Luong Tuan Thanh, Abhijit Sanyal · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.30924 · cited by 0 works (OpenAlex) · open-access PDF
Регулируемая оценка агентов посредством двухступенчатого тестирования и гарантии (REGATTA) — система измерений с тремя инструментами. 125-элементный перечень с цитируемыми источниками (5,400 сценариев) охватывает пять отраслевых клеток по юрисдикции в Соединенных Штатах и Вьетнаме. Регулируемая оценка агентов посредством двухступенчатого тестирования и гарантии (REGATTA) охватывает пять категорий. Регулируемая оценка агентов посредством двухступенчатого тестирования и гарантии — система измерений с тремя инструментами. Регулируемая проверка охватывает пять отраслевых клеток по юрисдикции в Соединенных Штатах и Вьетнаме
Machine translation (Meta M2M100); the English original follows.
Enterprise AI-agent evaluation needs evidence about whether test suites expose specified faults and whether their reference rules are trustworthy. We introduce REGATTA (Regulatory Evaluation of Agents via Two-stage Testing and Assurance), a measurement framework with three instruments. A 125-item source-cited checklist spans five industry-by-jurisdiction cells across the United States and Vietnam. A 25-fault corpus covers five categories, and a harness scores suite-level fault detection at the design and execution stages. We examine four measurement diagnostics on 180 generated suites (5,400 scenarios) from three LLM generators. A source audit finds 24 items misstating the rule in force, 46 additional citation defects, 31 items with only non-binding, unidentifiable, or unverifiable sources, and 24 verified clean. Only initially substance-flagged items received author review; all 24 clean rows received AI review only. Results therefore describe the instruments as written, not legal compliance. Item coverage varies, execution-stage fault profiles are stable across generators, and the historical stages agree on $66$--$69\%$ of fault--suite pairs. A current-model US-insurance retest finds stage disagreement above repeated-execution variation, but stage agreement $.536$ falls below pooled chance $.549$, failing V3's lower inequality. Blind adjudication by one expert supports the design-stage judging protocol ($κ=.72$, 95% interval $[.52,.88]$); 40 current-model execution probes give $κ=.75$. The instruments, audit, and reusable code are released.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Software engineering, Audit, Protocol (science), Checklist, Adjudication
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.30924, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange