Tianyi Lorena Yan, John Hewitt · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28303 · open-access PDF
언어 모델(LMs)의 체인-of-think (CoT) 모니터링은 AI 보안을위한 잘못된 행동을 감지하고 예방하는 핵심 전략입니다. 그러므로 모니터링 할 수없는 체인-of-think로 이어지는 강화 학습 (RL) 관행을 식별하는 것이 중요합니다. 그러한 관행 중 하나는 피할 수 없을 수 있습니다 : 보상 잘못된 정의. RL 환경이 복잡해지면서 완벽한 검증을 갖는 것은 불가능하며, 모델은 실제 작업 의도와는 다른 행동에 대해 보상됩니다. 우리는 보상 잘못된 정의만이 이미 모니터링 가능성을 악화시킨다는 것을 보여줍니다.
Machine translation (Meta M2M100); the English original follows.
Monitoring the chain-of-thought (CoT) of language models (LMs) is a key strategy for detecting and preventing misbehaviors for AI safety. As such, it is critical to identify reinforcement learning (RL) practices that lead to unmonitorable chain-of-thought. One such practice may be unavoidable: reward misspecification. As RL environments become more complex, it is infeasible to have perfect verification, and models are rewarded for behaviors that are different from the true task intent. We show that reward misspecification alone already degrades monitorability. We create a testbed with pairs of
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28303, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange