Newest papers

Auditing the Product, Not the Recursion: The Evaluation Evidence Behind Recursive Self-Improvement Claims

Zihan Chen, Di Zhu, Lei Nico Zheng · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.27610 · cited by 0 works (OpenAlex) · open-access PDF

Korean

재발행성 평가 보고서 확립 자기 개선 (RSI)는 자발적으로 진화하는 LLM 에이전트와 같은 AI 시스템을 더 나은 개선을 생산할 수 있습니다.이 약속을 평가하는 것은 최종 시스템에 대한 제품 주장을 분리하는 것이 필요하지만 개선 과정 자체에 대한 프로세스 주장을 설정합니다. 기존 설문 조사는 이러한 주장 설계에 대한 증거를 지정합니다. 얼마나 자주 연구는 증거가 측정되지 않는다고보고합니다. 우리는 재발행 가능한 arXiv 검색 프레임 프레임에있는 114 개의 유효한 연구의 전체 텍스트 인코딩을 통해이 질문을 검토합니다. 2026 년 10 월에 동일한 상태를 포함하여 90 개가 자신의 시스템을 RSI로 설명합니다. 코드북은 평가 설계에 대한 주장을 지정합니다. 독립적 인

Machine translation (Meta M2M100); the English original follows.

Abstract

Recursive self-improvement (RSI) promises AI systems, such as self-evolving LLM agents, that become better at producing further improvements. Evaluating this promise requires separating product claims about the final system from process claims about the improvement process itself. Existing surveys specify the evidence these claims require, but how often studies report that evidence remains unmeasured. We examine this question through full-text coding of 114 eligible studies in a reproducible arXiv search frame frozen in October 2026, including 90 that describe their own systems as RSI. The codebook links claims to evaluation designs. Independent human coding, third-reader adjudication, and a separate blind coding of improver comparisons assess coding reliability. Among the 90 RSI studies, 90% report product evaluation beyond development tasks under a broad criterion that includes partial or unstated separation, while 60% report evaluation on tasks unused for selection. Only 22% audit two or more reruns of the full loop. In a human-coded random sample, the RSI studies show the same contrast: 45% evaluate on tasks unused for selection and 14% audit two or more reruns. Of the 26 studies claiming a better improvement procedure, 23 report a related comparison, but only 4 compare earlier and later procedures from identical starting artifacts at a matched budget. Across these results and a comparison set of established works, evidence for repeatability and improved procedures remains less developed than product evaluation. Three propositions explain how selection effects, shared evaluator errors, and changes in system state limit what product evaluations can establish. Together, the findings support a claim-indexed reporting standard that connects each claim to the evaluation design needed to test it.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Audit, Computer science, Coding (social sciences), Product (mathematics), Process (computing), Set (abstract data type)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange