Xialie Zhuang, Mingcong Lei, Zunhai Su, Ri Liu, Jianjin Li, Peixian Ma, Jia Pan, Ngai Wong · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28522 · open-access PDF
Model Keputusan mendukung routing, guardrails, dan verifikasi output dalam sistem LLM. Namun, akurasi item saja tidak dapat menentukan apakah model membuat keputusan yang benar di antara variasi terkontrol atau memberikan kepercayaan yang dapat diandalkan pada prediksi mereka. Kami memperkenalkan DecisionPivot, benchmark diagnostik yang terdiri dari 48 tugas dan 40.033 item di tujuh peran keputusan. track intinya berisi 7.800 item dari 13 keluarga tugas yang dihasilkan, diatur menjadi kelompok kontras dengan jawaban komputasi yang dikonfirmasi oleh penyelesai independen. track referensi komplementer berisi 32.233 item dari 35
Machine translation (Meta M2M100); the English original follows.
Decision models support routing, guardrails, and output verification in LLM systems. However, item accuracy alone cannot establish whether models make correct decisions across controlled variations or assign reliable confidence to their predictions. We introduce DecisionPivot, a diagnostic benchmark comprising 48 tasks and 40,033 items across seven decision roles. Its core track contains 7,800 items from 13 generated task families, organized into contrast groups with computed answers verified by independently implemented solvers. The complementary reference track contains 32,233 items from 35
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28522, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange