Newest papers

Recoverability and Verifier Strength: Two Measured Preconditions for Unattended Agent Self-Improvement

Wang De, Peng Qi · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.27465 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

As plataformas de verificação de execução de agentes estão começando a melhorar os seus próprios agentes: eles estão lendo os traços de execução de um agente implantado, derivando uma mudança, tendo um verificador automatizado aprová-lo e liberando-o sem nenhum autor humano. Assim, quanto deste loop pode correr com segurança sem um humano? Proponemos duas quantidades enquanto uma plataforma pode medir. Verificar força $σ$ é o quão bem o verificador disponível recupera o comportamento - muitas vezes um juiz LLM - concorda com um verificador de referência confiável; ele limita o quanto um passo pode melhorar o agente, mas a mesma regra permite que a mesma ferramenta e as mesmas edições sejam executadas sem aviso e mantenham as atualizações de peso sob o julgamento humano. Nossa principal contribuição é anular

Machine translation (Meta M2M100); the English original follows.

Abstract

Agent platforms are starting to improve their own agents: they read a deployed agent's execution traces, derive a change, have an automated verifier approve it, and release it with no human author. How much of this loop can safely run without a human? We propose two quantities a platform can measure. Verifier strength $σ$ is how well the available verifier -- often an LLM judge -- agrees with a trusted reference verifier; it limits how much a step can improve the agent. Recoverability is how cheaply a bad change can be undone: whether undoing it needs a new verifier approval, and how much of the old behavior a rollback restores. A step should run unattended only if both are adequate. On our platform this rule allows prompt, tool and skill edits to run unattended and keeps weight updates under human sign-off. Our main contribution is these two definitions and how to measure them. We built a platform that applies the rule and tested it in two ways: on the platform's own approval gate, and in controlled experiments on the $τ^2$-bench agent benchmark. On verifier strength, the platform's LLM judge sometimes gave the same score no matter what it was shown (9 of 37 settings) and reported no error, so a verifier can look healthy while approving anything; its strength must be measured, and two simple tests catch the failure. On recoverability, restoring a saved copy brought the agent back to its original behavior, but a regenerated undo did not, so only changes whose old version is kept can be reversed without a human; and while a change was deployed, it altered the outcome of 17 percent of tasks beyond normal run-to-run variation, which no undo repairs. Together these support the rule: a change can run unattended only when its old version is kept and its verifier's strength has been measured.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Undo, Undoing, Computer science, Simple (philosophy), Computer security, Measure (data warehouse)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange