Luong Tuan Thanh, Abhijit Sanyal · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.30924 · cited by 0 works (OpenAlex) · open-access PDF
엔터프라이즈 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리전스 인텔리
Machine translation (Meta M2M100); the English original follows.
Enterprise AI-agent evaluation needs evidence about whether test suites expose specified faults and whether their reference rules are trustworthy. We introduce REGATTA (Regulatory Evaluation of Agents via Two-stage Testing and Assurance), a measurement framework with three instruments. A 125-item source-cited checklist spans five industry-by-jurisdiction cells across the United States and Vietnam. A 25-fault corpus covers five categories, and a harness scores suite-level fault detection at the design and execution stages. We examine four measurement diagnostics on 180 generated suites (5,400 scenarios) from three LLM generators. A source audit finds 24 items misstating the rule in force, 46 additional citation defects, 31 items with only non-binding, unidentifiable, or unverifiable sources, and 24 verified clean. Only initially substance-flagged items received author review; all 24 clean rows received AI review only. Results therefore describe the instruments as written, not legal compliance. Item coverage varies, execution-stage fault profiles are stable across generators, and the historical stages agree on $66$--$69\%$ of fault--suite pairs. A current-model US-insurance retest finds stage disagreement above repeated-execution variation, but stage agreement $.536$ falls below pooled chance $.549$, failing V3's lower inequality. Blind adjudication by one expert supports the design-stage judging protocol ($κ=.72$, 95% interval $[.52,.88]$); 40 current-model execution probes give $κ=.75$. The instruments, audit, and reusable code are released.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Software engineering, Audit, Protocol (science), Checklist, Adjudication
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.30924, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange