Newest papers

Over-Defense in Coding Agents Begins at Silent Forks

Hanjun Luo, Xuecheng Liu, Xiucheng Zhang, Zhuoning Xu, Yingbin Jin, Yankai Chen, Xue Liu, Hanan Salam · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.24949 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Model bahasa di belakang agen pengkodean didirikan dengan penghargaan hasil, dan metode baru-baru ini berkonsentrasi kredit pada putaran yang menentukan hasil. Kami menemukan bahwa tindakan pertahanan melampaui apa yang dibutuhkan tugas melarikan diri keduanya. Mereka dimulai pada fork diam, putaran di mana setiap tindakan berikutnya mengarah ke hasil yang sama. Tindakan tersebut meninggalkan tingkat keberhasilan yang sama dan perilaku yang tidak berubah, dan model perbatasan mengambil cabang yang tidak proporsional di sana hingga 91% dari tindakan yang menentukan di bawah anggaran implisit. Perbaikan yang paling langsung, seorang pemeriksa perilaku dalam reward trajectory, mencapai fork ini hanya melalui trajectories yang sepenuhnya proporsional, meninggalkan 74% dari kelompok GRPO dengan penghargaan yang sama dan perilaku yang tidak berubah. Ini memotivasi pemikiran ulang sinyal belajar

Machine translation (Meta M2M100); the English original follows.

Abstract

Language models behind coding agents are post-trained with outcome rewards, and recent methods concentrate credit on the turns that decide the outcome. We find that defensive actions beyond what a task requires escape both. They begin at silent forks, the turns where every next action leads to the same outcome. Such actions leave the success rate unchanged, and frontier models take the disproportionate branch there in up to 91% of decisive actions under implicit budgets. The most direct fix, a checker in the trajectory reward, reaches these forks only through fully proportionate trajectories, leaving 74% of GRPO groups with identical rewards and behavior unchanged. This motivates a rethink of where learning signals belong, namely where behavior diverges, which a behavioral checker can locate and the outcome cannot. Our method, ForkDPO, contrasts a proportionate and a disproportionate action at the same prefix, at silent forks on both sides of the required behavior. On a test set of unseen repositories, it lowers a 4B model's disproportionate behavior from 70.3% to 58.9%, against 67.7% for DPO at randomly chosen turns, establishing behavioral divergence as a learning signal for how agents act beyond whether they finish. The code is publicly available at https://github.com/Astarojth/silent-forks.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Outcome (game theory), Coding (social sciences), Task (project management), Set (abstract data type), Action (physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange