Newest papers

VeriEvolve: Evaluator-First Evolution for Tool-Using Agents

Yueshen Li, Kamer Ali Yüksel, Gökhan Tür, Dilek Z. Hakkani-Tur, João Pedro Maia · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29005 · cited by 0 works (OpenAlex) · open-access PDF

Korean

스스로 개선하는 에이전트에 대한 중앙 장애물은 현실적인 상태 작업에 대한 신뢰할 수 있는 피트니스 신호가 없다는 것입니다. 우리는 실현자 첫 번째 진화 프레임 워크인 \textsc{VeriEvolve}를 소개합니다. 스스로 개선하는 에이전트를 최적화하기 전에이 실패한 평가 레이어를 구축합니다. 작업 지침에서 VeriEvolve는 지침 기반의 도전 상태와 실행 가능한 평가기를 합성합니다. 작업 흐름 진화의 각 라운드 동안, 평가자는 냉동되고 피트니스 신호로 봉사합니다. 라운드 사이에, VeriEvolve는 실행자에 의해 생성된 리머에 대한 최종 상태를 변환함으로써 평가자를 수정합니다. 실행자 연구에 대한 검색 결과는 30 MCPM Filark

Machine translation (Meta M2M100); the English original follows.

Abstract

A central obstacle to self-improving agents is the absence of a reliable fitness signal for realistic stateful tasks. We introduce \textsc{VeriEvolve}, an evaluator-first evolution framework that constructs this missing evaluation layer before optimizing the agent. From task instruction, VeriEvolve synthesizes instruction-grounded challenge states and an executable evaluator that checks them. During each round of workflow evolution, the evaluator is frozen and serves as the fitness signal. Between rounds, VeriEvolve revises the evaluator by converting the end states of strong and weak runs into a pair of counterfactual test cases. Across all 30 MCPMark Filesystem tasks, VeriEvolve constructs 30 evaluators with 141 challenge states, achieving 141/141 agreement on the generated suites and 113/122 agreement on held-out states produced by executor agents. Evaluator-guided evolution increases externally verified success from 6/30 to 18/30 for GLM-5.2 and from 27/30 to 30/30 for Claude; refinement removes all three false positives that the weak executor exposes. Cross-environment studies on PostgreSQL, Playwright, Notion, and GitHub further expose complementary limits from observability, state representation, and search/execution capacity. These results show that executable evaluators can be synthesized and used to guide search when task success is observable and an external verifier is available for calibration.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Executor, Executable, Computer science, Task (project management), Counterfactual thinking, Stateful firewall

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange