Newest papers

VeriEvolve: Evaluator-First Evolution for Tool-Using Agents

Yueshen Li, Kamer Ali Yüksel, Gökhan Tür, Dilek Z. Hakkani-Tur, João Pedro Maia · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29005 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

对于自我改进的代理人来说,一个核心障碍是缺乏可靠的健身信号来执行现实状态任务。我们引入了“textsc{VeriEvolve}”,一个任务评估器第一进化框架,在优化代理人之前构建了这种缺失的评估层。从任务指令中,VeriEvolve将基于指令的挑战状态和可执行的评估器合成为一对。在每个工作流程演变中,评估器被冻结并作为健身信号。在轮流之间,VeriEvolve将评估器通过将强和弱的状态转换为一对反事实的测试案例。在所有30个MCPM Filarkes任务中,VeriEvolve构建了30个评估器,具有141个挑战状态,实现了141/14

Machine translation (Meta M2M100); the English original follows.

Abstract

A central obstacle to self-improving agents is the absence of a reliable fitness signal for realistic stateful tasks. We introduce \textsc{VeriEvolve}, an evaluator-first evolution framework that constructs this missing evaluation layer before optimizing the agent. From task instruction, VeriEvolve synthesizes instruction-grounded challenge states and an executable evaluator that checks them. During each round of workflow evolution, the evaluator is frozen and serves as the fitness signal. Between rounds, VeriEvolve revises the evaluator by converting the end states of strong and weak runs into a pair of counterfactual test cases. Across all 30 MCPMark Filesystem tasks, VeriEvolve constructs 30 evaluators with 141 challenge states, achieving 141/141 agreement on the generated suites and 113/122 agreement on held-out states produced by executor agents. Evaluator-guided evolution increases externally verified success from 6/30 to 18/30 for GLM-5.2 and from 27/30 to 30/30 for Claude; refinement removes all three false positives that the weak executor exposes. Cross-environment studies on PostgreSQL, Playwright, Notion, and GitHub further expose complementary limits from observability, state representation, and search/execution capacity. These results show that executable evaluators can be synthesized and used to guide search when task success is observable and an external verifier is available for calibration.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Executor, Executable, Computer science, Task (project management), Counterfactual thinking, Stateful firewall

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange