Xialie Zhuang, Mingcong Lei, Zunhai Su, Ri Liu, Jianjin Li, Peixian Ma, Jia Pan, Ngai Wong · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28522 · open-access PDF
决策模型在LLM系统中支持路由、防护栏和输出验证,但单靠项目准确性无法确定模型是否在受控的变异中做出正确的决定,或对其预测赋予可靠的信心。我们引入了DocisionPivot,一个诊断基准,包括48个任务和40033个项目在七个决策角色中。其核心轨道包含13个生成的任务家族的7800个项目,分组成对比组,由独立实施的解决方案者验证的计算答案。
Machine translation (Meta M2M100); the English original follows.
Decision models support routing, guardrails, and output verification in LLM systems. However, item accuracy alone cannot establish whether models make correct decisions across controlled variations or assign reliable confidence to their predictions. We introduce DecisionPivot, a diagnostic benchmark comprising 48 tasks and 40,033 items across seven decision roles. Its core track contains 7,800 items from 13 generated task families, organized into contrast groups with computed answers verified by independently implemented solvers. The complementary reference track contains 32,233 items from 35
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28522, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange