Newest papers

Asymmetric On-Policy Distillation Learns In-Context Exploration

Sriyash Poddar, Yanda Bao, Jacob Krantz, Matthew Chang, Xavier Puig, Roozbeh Mottaghi, Natasha Jaques, Abhishek Gupta · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30105 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

A tomada de decisões de transferência de visão seqüencial sob observação parcial é difícil porque os agentes devem explorar para inferir informações fixas, relevantes para tarefas antes de agir efetivamente. Aprender tais comportamentos de coleta de informações tabula rasa através de aprendizagem de reforço on-line é amostra-ineficiente, enquanto a imitação offline media ações de especialistas em estados ocultos e falha em aprender a explorar. Neste trabalho, propomos ASTEROID, um quadro de aprendizagem simples e eficaz estudante-professor supervisionado que pode aprender a exploração em tempo de teste. Embora um especialista privilegiado nunca precise explorar, demonstramos que um aluno condicionado pela história pode aprender a fazê-lo quando treinado com contextos de política: o primeiro aluno coleta sobre contextos de política sob observação parcial e

Machine translation (Meta M2M100); the English original follows.

Abstract

Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and the privileged expert provides action labels conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinforcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Observability, Exploit, Artificial intelligence, Reinforcement learning, Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange