Newest papers

Over-Defense in Coding Agents Begins at Silent Forks

Hanjun Luo, Xuecheng Liu, Xiucheng Zhang, Zhuoning Xu, Yingbin Jin, Yankai Chen, Xue Liu, Hanan Salam · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.24949 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

コードエージェントの背後にある言語モデルは、結果の報酬を後で訓練され、最近の方法は、結果を決定するターンにクレジットを集中する。私たちは、課題が要求するもの以外の防御行動が両方から脱出することを発見します。 彼らは静かなフォークで始まり、次のすべてのアクションが同じ結果につながるターンです。 そのようなアクションは、同じ報酬と行動を有するGRPOグループの74%を変更せず、境界モデルは、明示的な予算の下で決定的なアクションの91%まで不均衡な分野を取ります。 最も直接的な修正は、行動チェックアウトは、報酬の軌道にあり、これらのフォークに到達します。 私たちの方法、FDPOorkは、

Machine translation (Meta M2M100); the English original follows.

Abstract

Language models behind coding agents are post-trained with outcome rewards, and recent methods concentrate credit on the turns that decide the outcome. We find that defensive actions beyond what a task requires escape both. They begin at silent forks, the turns where every next action leads to the same outcome. Such actions leave the success rate unchanged, and frontier models take the disproportionate branch there in up to 91% of decisive actions under implicit budgets. The most direct fix, a checker in the trajectory reward, reaches these forks only through fully proportionate trajectories, leaving 74% of GRPO groups with identical rewards and behavior unchanged. This motivates a rethink of where learning signals belong, namely where behavior diverges, which a behavioral checker can locate and the outcome cannot. Our method, ForkDPO, contrasts a proportionate and a disproportionate action at the same prefix, at silent forks on both sides of the required behavior. On a test set of unseen repositories, it lowers a 4B model's disproportionate behavior from 70.3% to 58.9%, against 67.7% for DPO at randomly chosen turns, establishing behavioral divergence as a learning signal for how agents act beyond whether they finish. The code is publicly available at https://github.com/Astarojth/silent-forks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Outcome (game theory), Coding (social sciences), Task (project management), Set (abstract data type), Action (physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange