Newest papers

VaDar: Implicit Goal Conditioning via Value Disaggregation

Shashwat Saxena, Mehul Goel, Sreyas Venkataraman, Sarvesh Patil, Max Simchowitz · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26629 · cited by 0 works (OpenAlex) · open-access PDF

Korean

장거리 임무는 현대 인공지능과 로봇 기술에 중요하지만, 강화 학습에 도전을 제기하는 도전이다. 역사적으로, 대중적인 접근 방식은 임무를 하위 목표로 분해하고, 각 하위 작업을 연속적으로 수행하는 하위 목표 조건 정책을 배우는 것이었습니다. 우리는 가치 분해 (VaDar)라고 불리는 대안을 도입합니다. 또한, 단일 하위 목표 독립적 인 정책을 훈련하여 하위 목표 비판적 가치의 선형 조합을 극대화합니다. 하위 목표 정보는 훈련 시간에만 필요하기 때문에, VaDar는 가능한 한 비용이 많이 들거나 어려운 하위 목표 선택의 필요성을 제거합니다. 또한, 신중한 실험을 통해, 우리는 VaDar (1)가 항상 경쟁력 있고 종종 목표 조건과

Machine translation (Meta M2M100); the English original follows.

Abstract

Long-horizon tasks are important for modern AI and robotics, yet pose a challenge for reinforcement learning. Historically, a popular approach has been to decompose a task into subgoals, and learn a subgoal-conditioned policy to execute each subtask in sequence. We introduce an alternative called Value Disaggregation (VaDar), which trains a single subgoal-independent policy to maximize a linear combination of per-subgoal critic values. Because subgoal information is needed only at training-time, VaDar removes the need for possibly costly or cumbersome subgoal-selection during policy deployment. Moreover, through careful experiments, we show that VaDar (1) is always competitive with, and frequently outperforms, natural baselines like goal-conditioning and (2) succeeds on tasks with non-sequential goal structure and multiple optimization objectives. In particular, VaDar is the first algorithm to solve challenging, long-horizon tasks in the RoboCasa suite, even when starting from a policy pretrained from behavior cloning with near-zero initial task success. Taken together, VaDar's successes suggest that subgoal information benefits actor-critic reinforcement learning by enhancing the efficacy of critic learning, whereas policy goal-conditioning is often unnecessary.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Task (project management), Computer science, Artificial intelligence, Value (mathematics), Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange