Tianyi Lorena Yan, John Hewitt · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28303 · open-access PDF
监测语言模型(LMs)的思维链(CoT)是检测和预防AI安全的不良行为的关键策略,因此,确定导致不可监测的思维链(RL)的增强学习(Reinforcement Learning)做法至关重要。其中一种做法可能是不可避免的:奖励误定。随着RL环境变得更加复杂,没有完美的验证是不可行的,并且模型因与真正的任务意图不同而受到奖励。
Machine translation (Meta M2M100); the English original follows.
Monitoring the chain-of-thought (CoT) of language models (LMs) is a key strategy for detecting and preventing misbehaviors for AI safety. As such, it is critical to identify reinforcement learning (RL) practices that lead to unmonitorable chain-of-thought. One such practice may be unavoidable: reward misspecification. As RL environments become more complex, it is infeasible to have perfect verification, and models are rewarded for behaviors that are different from the true task intent. We show that reward misspecification alone already degrades monitorability. We create a testbed with pairs of
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28303, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange