Newest papers

VaDar: Implicit Goal Conditioning via Value Disaggregation

Shashwat Saxena, Mehul Goel, Sreyas Venkataraman, Sarvesh Patil, Max Simchowitz · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26629 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Las tareas de largo horizonte son importantes para la IA y la robótica moderna, pero presentan un desafío para el aprendizaje de reforzamiento. Históricamente, un enfoque popular ha sido descomponer una tarea en subobjetivos, y aprender una política condicionada por subobjetivos para ejecutar cada subtarea en secuencia. Introducimos una alternativa llamada Value Disaggregation (VaDar), que trabaja una única política independiente de subobjetivos para maximizar una combinación lineal de valores críticos por subobjetivo. Debido a que la información subobjetiva es necesaria sólo en el tiempo de entrenamiento, VaDar elimina la necesidad de una selección subobjetiva potencialmente costosa o complicada durante la implementación de la política. Además, a través de experimentos cuidadosos, mostramos que Va

Machine translation (Meta M2M100); the English original follows.

Abstract

Long-horizon tasks are important for modern AI and robotics, yet pose a challenge for reinforcement learning. Historically, a popular approach has been to decompose a task into subgoals, and learn a subgoal-conditioned policy to execute each subtask in sequence. We introduce an alternative called Value Disaggregation (VaDar), which trains a single subgoal-independent policy to maximize a linear combination of per-subgoal critic values. Because subgoal information is needed only at training-time, VaDar removes the need for possibly costly or cumbersome subgoal-selection during policy deployment. Moreover, through careful experiments, we show that VaDar (1) is always competitive with, and frequently outperforms, natural baselines like goal-conditioning and (2) succeeds on tasks with non-sequential goal structure and multiple optimization objectives. In particular, VaDar is the first algorithm to solve challenging, long-horizon tasks in the RoboCasa suite, even when starting from a policy pretrained from behavior cloning with near-zero initial task success. Taken together, VaDar's successes suggest that subgoal information benefits actor-critic reinforcement learning by enhancing the efficacy of critic learning, whereas policy goal-conditioning is often unnecessary.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Task (project management), Computer science, Artificial intelligence, Value (mathematics), Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange