Newest papers

Do Coding Agents Understand Why Systems Crash---and Why They Don't?

Chenxi Huang, Haonan Wang, Jinghe Zhang, Aleksandr Nogikh, Franjo Ivančić, Petros Maniatis, Eugene Wu, Kostis Kaffes, Junfeng Yang, Baishakhi Ray · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29332 · cited by 0 works (OpenAlex) · open-access PDF

French

Les agents de crash sont de plus en plus capables de résoudre les bugs du monde réel, mais ils luttent encore avec de grandes bases de codes de systèmes telles que le noyau Linux. Pour comprendre pourquoi, nous testons comment les agents raisonnent l'exécution dans ces bases de codes à travers le raisonnement de crash: prédisant à partir du code source si une entrée donnée s'effondre dans le système. Une prédiction correcte seule, cependant, n'établit pas cette capacité, car elle peut provenir de la simple détection de code suspect. Dans ce document, nous proposons des interventions de source: nous évaluons les principaux résultats de trois arbres source qui diffèrent par un seul compromis, de sorte que la même entrée ne s'effondre qu'un

Machine translation (Meta M2M100); the English original follows.

Abstract

Coding agents are increasingly capable of fixing real-world bugs, yet they still struggle with large systems codebases such as the Linux kernel. To understand why, we test how agents reason about execution in these codebases through crash reasoning: predicting from source code whether a given input crashes the system. A correct prediction alone, however, does not establish this ability, since it may come from merely spotting suspicious code. In this paper, we propose source interventions: we pair source trees that differ by a single commit, so that the same input crashes only one of them, and test whether agents predict crashes more often on the crashing tree. Experiments with four coding agents on SysRB, our dataset of 934 such pairs from Linux and OSS-Fuzz projects, yield three main findings. First, agents are biased toward predicting no crash: they rarely raise false alarms but miss many real crashes, and even the best agent misses over a quarter of them. Second, locating the code that determines the crash outcome is not enough: agents often misjudge its effect, especially when the crash occurs far from that code. Third, testing only on crashing code is misleading: agents that merely notice suspicious code predict crashes on both source trees, which would look like correct reasoning on the crashing tree alone. Together, these findings indicate that current coding agents cannot yet reliably reason about execution in complex systems codebases, and that paired evaluation is necessary to measure this ability.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Source code, Crash, Codebase, Notice, Coding (social sciences)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange