Newest papers

VaDar: Implicit Goal Conditioning via Value Disaggregation

Shashwat Saxena, Mehul Goel, Sreyas Venkataraman, Sarvesh Patil, Max Simchowitz · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26629 · cited by 0 works (OpenAlex) · open-access PDF

Hindi

दीर्घकालिक कार्य आधुनिक एआई और रोबोटिक्स के लिए महत्वपूर्ण हैं, फिर भी मजबूत सीखने के लिए एक चुनौती पैदा करते हैं। ऐतिहासिक रूप से, एक लोकप्रिय दृष्टिकोण एक कार्य को उपचुनाव में विभाजित करना है, और एक अनुक्रम में प्रत्येक उपचुनाव को निष्पादित करने के लिए एक उपचुनाव की शर्त रखने वाली नीति सीखना है। हम एक वैकल्पिक को Value Disaggregation (VaDar) कहते हैं, जो एक एकल उपचुनाव स्वतंत्र नीति को प्रशिक्षित करता है ताकि प्रति उपचुनाव महत्वपूर्ण मूल्यों के एक लाइनर संयोजन को अधिकतम किया जा सके। चूंकि उपचुनाव की जानकारी केवल प्रशिक्षण समय पर आवश्यक है, VaDar नीति के कार्यान्वयन के दौरान संभवतः महंगी या कठिन उपचुनाव

Machine translation (Meta M2M100); the English original follows.

Abstract

Long-horizon tasks are important for modern AI and robotics, yet pose a challenge for reinforcement learning. Historically, a popular approach has been to decompose a task into subgoals, and learn a subgoal-conditioned policy to execute each subtask in sequence. We introduce an alternative called Value Disaggregation (VaDar), which trains a single subgoal-independent policy to maximize a linear combination of per-subgoal critic values. Because subgoal information is needed only at training-time, VaDar removes the need for possibly costly or cumbersome subgoal-selection during policy deployment. Moreover, through careful experiments, we show that VaDar (1) is always competitive with, and frequently outperforms, natural baselines like goal-conditioning and (2) succeeds on tasks with non-sequential goal structure and multiple optimization objectives. In particular, VaDar is the first algorithm to solve challenging, long-horizon tasks in the RoboCasa suite, even when starting from a policy pretrained from behavior cloning with near-zero initial task success. Taken together, VaDar's successes suggest that subgoal information benefits actor-critic reinforcement learning by enhancing the efficacy of critic learning, whereas policy goal-conditioning is often unnecessary.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Reinforcement learning, Task (project management), Computer science, Artificial intelligence, Value (mathematics), Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange