Newest papers

AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation

Likui Zhang, Zhe Zhu, Zhihao Zhan, Yibo Bai, Hanlin Wang, Dongwei Li, Bin Wang, Tianle Zhang, Xiaodan Liang, Liang Lin · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30097 · cited by 0 works (OpenAlex) · open-access PDF

Korean

비전-언어-행동 (VLA) 모델은 로봇 조작에서 약속을 보여 왔지만 모방 학습 (IL)은 단독으로 복잡한 장거리 기술을 마스터하기 위해 종종 어려움을 겪습니다. 성능 강화 학습 (RL)은 VLA 정책을 더욱 향상시키는 효과적인 경로를 제공합니다. 그러나 기존 VLA-RL 방법은 일반적으로 경로 수준의 균형없는 보상으로 인한 지연 및 편리한 신용 할당으로 인해 고통을 겪습니다. 이 불균형한 하위 작업 마스터링은 전체 성능을 제한하는 병아리를 만듭니다. 이러한 도전을 해결하기 위해 우리는 AtomicRL을 제안합니다. 현재의 VLA-RL 방법은 신용 할당 및 적응성 탐구를위한 원자적 인 작업 구조를

Machine translation (Meta M2M100); the English original follows.

Abstract

Vision-language-action (VLA) models have shown promise in robotic manipulation, but imitation learning (IL) alone often struggles to master complex long-horizon skills. Reinforcement learning (RL) offers an effective avenue for further improving VLA policies. However, existing VLA-RL methods typically suffer from delayed and biased credit assignment due to trajectory-level sparse rewards, while imbalanced subtask mastery creates bottlenecks that limit overall performance. To address these challenges, we propose AtomicRL, an RL post-training framework that leverages atomic task structures for reliable credit assignment and adaptive exploration. AtomicRL introduces Semantic-Aware Policy Optimization (SAPO), which uses semantic-aware gating to constrain TD residuals from failed atomic subtasks from propagating across task boundaries, mitigating credit contamination while preserving intra-subtask credit assignment and value bootstrapping. In addition, Bottleneck-Adaptive Target Exploration (BATE) dynamically coordinates task sampling and local exploration incentives based on the mastery of individual atomic subtasks, directing learning toward the remaining bottlenecks. Finally, we develop a Real2Sim2Real pipeline that bridges online RL in simulation with real-world deployment, facilitating practical transfer to physical robots. On LIBERO-LONG, AtomicRL achieves absolute performance gains of 3.6% and 4.0% over PPO across two VLA architectures. On CALVIN ABC-D, it improves full-task performance by up to 9.5% over PPO. Combined with the Real2Sim2Real pipeline, AtomicRL further improves real-world task performance by an average of 21.3%. These results demonstrate that AtomicRL effectively improves the learning efficiency and manipulation performance of VLA policies on long-horizon tasks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Computer science, Task (project management), Artificial intelligence, Pipeline (software), Imitation

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange