Newest papers

VaDar: Implicit Goal Conditioning via Value Disaggregation

Shashwat Saxena, Mehul Goel, Sreyas Venkataraman, Sarvesh Patil, Max Simchowitz · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26629 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Tugas jangka panjang penting bagi AI dan robotika modern, namun merupakan tantangan bagi pembelajaran penguatan. Secara historis, pendekatan yang populer adalah untuk memecah tugas menjadi subtarget, dan mempelajari kebijakan subtarget yang dikondisikan untuk melakukan setiap subtask secara berurutan. Kami memperkenalkan alternatif yang disebut Value Disaggregation (VaDar), yang melatih kebijakan independen subtarget tunggal untuk memaksimalkan kombinasi linier nilai kritis per subtarget. Karena informasi subtarget diperlukan hanya pada waktu pelatihan, VaDar menghilangkan kebutuhan untuk seleksi subtarget yang mungkin mahal atau rumit selama implementasi kebijakan. Selain itu, melalui percobaan yang hati-hati, kami menunjukkan bahwa VaDar (1) selalu kompetitif dengan, dan sering melampaui, garis dasar alami seperti target-kondisi dan (2) berhasil dalam tugas dengan struktur tujuan non-sequential dan

Machine translation (Meta M2M100); the English original follows.

Abstract

Long-horizon tasks are important for modern AI and robotics, yet pose a challenge for reinforcement learning. Historically, a popular approach has been to decompose a task into subgoals, and learn a subgoal-conditioned policy to execute each subtask in sequence. We introduce an alternative called Value Disaggregation (VaDar), which trains a single subgoal-independent policy to maximize a linear combination of per-subgoal critic values. Because subgoal information is needed only at training-time, VaDar removes the need for possibly costly or cumbersome subgoal-selection during policy deployment. Moreover, through careful experiments, we show that VaDar (1) is always competitive with, and frequently outperforms, natural baselines like goal-conditioning and (2) succeeds on tasks with non-sequential goal structure and multiple optimization objectives. In particular, VaDar is the first algorithm to solve challenging, long-horizon tasks in the RoboCasa suite, even when starting from a policy pretrained from behavior cloning with near-zero initial task success. Taken together, VaDar's successes suggest that subgoal information benefits actor-critic reinforcement learning by enhancing the efficacy of critic learning, whereas policy goal-conditioning is often unnecessary.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Task (project management), Computer science, Artificial intelligence, Value (mathematics), Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange