Xu Li, Simon Yu, Minzhou Pan, Yiyou Sun, Zhaorun Chen, Bo Li, Weiyan Shi · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28934 · open-access PDF
Agen pengkodean sekarang menyelesaikan tugas secara otonom, memodifikasi keadaan dunia nyata seperti artefak, lingkungan perangkat lunak, dan konfigurasi di sepanjang jalan. Namun, mereka menunjukkan kesadaran cadangan yang terbatas: mengakui bahwa suatu tindakan akan memodifikasi keadaan yang mungkin dibutuhkan kemudian, dan menyimpannya sebelumnya. Untuk mengevaluasi kesadaran cadangan, kami memperkenalkan RegretBench, sebuah benchmark dari 205 tugas yang dapat dijalankan yang didasarkan pada saat-saat penyesalan yang dilaporkan pada platform seperti X.com, GitHub, dan Stack Overflow, di mana pengguna secara eksplisit menyesal tidak menyimpan keadaan yang mereka butuhkan kemudian.
Machine translation (Meta M2M100); the English original follows.
Coding agents now complete tasks autonomously, modifying real-world state such as artifacts, software environments, and configurations along the way. Yet they show limited backup awareness: recognizing that an action will modify state that may later be needed, and preserving it beforehand. To evaluate backup awareness, we introduce RegretBench, a benchmark of 205 executable tasks grounded in regret moments reported on platforms such as X.com, GitHub, and Stack Overflow, where users explicitly regretted not preserving state they later needed. Across six frontier models, agents complete 86.1% of
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28934, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange