Mahdi Sabbaghi, George Pappas, Hamed Hassani, Adel Javanmard · 2026 · arXiv
DOI: 10.48550/arXiv.2610.31563 · open-access PDF
在线政策蒸发(OPD)通常通过直接从学生身上采样的轨道进行培训来优化像监督精细调(SFT)这样的离线方法,但是标准OPD训练学生在轨道的每个状态下跟随教师,假设教师指导总是有益的。我们相反地区分教师喜欢什么,以及实际上帮助当前学生取得成功的东西。
Machine translation (Meta M2M100); the English original follows.
Online Policy Distillation (OPD) often outperforms offline methods such as Supervised Fine-Tuning (SFT) by training on trajectories sampled directly from the student. However, standard OPD trains the student to follow the teacher at every state of the trajectory, assuming that teacher guidance is always beneficial. We instead distinguish between what the teacher prefers and what actually helps the current student succeed. We introduce Advantage-Based On-Policy Distillation (AOPD), which weights teacher actions by their counterfactual advantage under the student. Teacher actions that improve th
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.31563, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange