Hongyu Li, Xinlong Huang, Zhan Yu, Jing Zhao, Zhe Zhao, Juntao Zhong, Xiong Kuang, Rui Yuan · 2026 · arXiv
DOI: 10.48550/arXiv.2610.29225 · open-access PDF
目前的LLM安全基准主要依赖于二进制安全或不安全的标签,缺乏解释深度,并在自动化评判中缺乏一致性。 为了解决这一问题,我们引入了DiagSafe,一种超越二进制分类的诊断评估框架,通过使用快速级别的分类。 根据全球适用的AI安全政策,每个分类都提供了两个维度的明确标准:安全,旨在针对具体的伤害,有特殊的豁免来抑制虚假的阳性,以及有用性,以独立评估指令的实现。
Machine translation (Meta M2M100); the English original follows.
Current LLM safety benchmarks predominantly rely on binary safe or unsafe labels, lacking explanatory depth and suffering from low consistency across automated judges. To address this, we introduce DiagSafe, a diagnostic evaluation framework that moves beyond binary classification by employing prompt-level rubrics. Grounded in globally applicable AI safety policies, each rubric provides explicit criteria along two dimensions: safety, which targets concrete harms with specific exemptions to suppress false positives, and usefulness, which independently evaluates instruction fulfillment. To ensur
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.29225, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange