Newest papers

Asymmetric On-Policy Distillation Learns In-Context Exploration

Sriyash Poddar, Yanda Bao, Jacob Krantz, Matthew Chang, Xavier Puig, Roozbeh Mottaghi, Natasha Jaques, Abhishek Gupta · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30105 · cited by 0 works (OpenAlex) · open-access PDF

French

L'apprentissage de tels comportements de collecte d'informations tabula rasa via l'apprentissage de renforcement en ligne est échantillon-inefficace, tandis que l'imitation hors ligne moyenne les actions d'experts dans des états cachés et ne parvient pas à apprendre l'exploration. Dans ce travail, nous proposons ASTEROID, un cadre d'apprentissage simple et efficace étudiant-enseignant supervisé qui peut apprendre l'exploration en temps de test. Bien qu'un expert privilégié n'ait jamais besoin d'explorer, nous démontrons qu'un étudiant conditionné par l'histoire peut apprendre à le faire lorsqu'il est formé avec des contextes de politique: l'étudiant recueille le contexte de politique en temps partiel

Machine translation (Meta M2M100); the English original follows.

Abstract

Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and the privileged expert provides action labels conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinforcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Observability, Exploit, Artificial intelligence, Reinforcement learning, Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange