Hossein A. Rahmani, Bin Wu, Chaeeun Lee, Fanghua Ye, Sahan Bulathwela, Sujay Kumar Jauhar, Marzieh Fadaee, Nick Craswell · 2026 · arXiv
DOI: 10.48550/arXiv.2610.26679 · open-access PDF
에이전트의 계획 능력을 평가하는 것은 여전히 오픈 도메인 작업에 도전적이며, 실행을 통해 절차를 검증하는 것은 종종 비현실적입니다. 기존의 평가는 일반적으로 최종 계획만을 평가하고 실수, 부적절 또는 잘못된 행동과 같은 절차 오류를 구별하지 않습니다. STEPBENCH는 81 개의 오픈 도메인 카테고리에 걸쳐 wikiHow에서 검색한 6,885 개의 검증 절차의 기준을 소개합니다. 각 절차에서 우리는 5 개의 진단 작업을 추출합니다.
Machine translation (Meta M2M100); the English original follows.
Evaluating an agent's planning ability remains challenging for open-domain tasks, where verifying procedures through execution is often impractical. Existing evaluations typically assess only the final plan and do not distinguish procedural errors such as missing, incorrect, or misordered actions. We introduce STEPBENCH, a benchmark of 6,885 verified procedures crawled from wikiHow across 81 open-domain categories. From each procedure, we derive 5 diagnostic tasks: four with deterministic answers involving withheld, incorrect, or shuffled steps, and one requiring next-step generation. We addit
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26679, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange