Newest papers

Asymmetric On-Policy Distillation Learns In-Context Exploration

Sriyash Poddar, Yanda Bao, Jacob Krantz, Matthew Chang, Xavier Puig, Roozbeh Mottaghi, Natasha Jaques, Abhishek Gupta · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30105 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Pengambilan keputusan yang efisien dalam pengamatan parsial adalah sulit karena agen harus mengeksplorasi untuk menyimpulkan informasi yang tersembunyi, yang relevan dengan tugas sebelum bertindak secara efektif. Belajar perilaku pengumpulan informasi seperti tabula rasa melalui pembelajaran penguatan online adalah tidak efisien, sementara imitasi offline rata-rata tindakan ahli di berbagai negara yang tersembunyi dan gagal untuk mempelajari eksplorasi. Dalam pekerjaan ini, kami mengusulkan ASTEROID, kerangka kerja pembelajaran yang sederhana dan efektif di bawah pengawasan siswa dan guru yang dapat mempelajari eksplorasi waktu uji coba. Meskipun seorang ahli istimewa tidak pernah perlu mengeksplorasi, kami menunjukkan bahwa seorang siswa yang berbasis sejarah dapat belajar melakukannya ketika dilatih dengan konteks kebijakan: siswa pertama mengumpulkan konteks kebijakan di bawah observasi parsial, dan ahli istimewa memberikan label tindakan yang diatur

Machine translation (Meta M2M100); the English original follows.

Abstract

Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and the privileged expert provides action labels conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinforcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-denied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Observability, Exploit, Artificial intelligence, Reinforcement learning, Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange