Xialie Zhuang, Mingcong Lei, Zunhai Su, Ri Liu, Jianjin Li, Peixian Ma, Jia Pan, Ngai Wong · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28522 · open-access PDF
Los modelos de decisión soportan el enrutamiento, los guardrails y la verificación de la salida en los sistemas LLM. Sin embargo, la exactitud de los elementos por sí sola no puede determinar si los modelos toman decisiones correctas a través de variaciones controladas o asignan confianza fiable a sus predicciones. Introducimos DecisionPivot, un benchmark de diagnóstico que incluye 48 tareas y 40.033 elementos a través de siete roles de decisión. Su pista principal contiene 7.800 artículos de 13 familias de tareas generadas, organizados en grupos de contraste con respuestas computadas verificadas por solucionadores implementados de forma independiente.
Machine translation (Meta M2M100); the English original follows.
Decision models support routing, guardrails, and output verification in LLM systems. However, item accuracy alone cannot establish whether models make correct decisions across controlled variations or assign reliable confidence to their predictions. We introduce DecisionPivot, a diagnostic benchmark comprising 48 tasks and 40,033 items across seven decision roles. Its core track contains 7,800 items from 13 generated task families, organized into contrast groups with computed answers verified by independently implemented solvers. The complementary reference track contains 32,233 items from 35
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28522, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange