Hongyu Li, Xinlong Huang, Zhan Yu, Jing Zhao, Zhe Zhao, Juntao Zhong, Xiong Kuang, Rui Yuan · 2026 · arXiv
DOI: 10.48550/arXiv.2610.29225 · open-access PDF
現在のLLMのセキュリティ基準は、主にバイナリの安全または不安全なラベルに依存し、説明深さが欠け、自動化された審査員の間で低い一貫性に苦しんでいます。これに対処するために、我々はバイナリの分類を超える診断評価枠組みであるDiagSafeを導入し、プロンプトレベルの分類を採用しています。世界的に適用されるAIのセキュリティポリシーに基づいて、各分類は2つの次元に沿って明確な基準を提供します:安全性、特定の例外を含む具体的な害を標的とし、誤ったポジティブを抑制するために、および有用性、指示の達成を独立して評価します。
Machine translation (Meta M2M100); the English original follows.
Current LLM safety benchmarks predominantly rely on binary safe or unsafe labels, lacking explanatory depth and suffering from low consistency across automated judges. To address this, we introduce DiagSafe, a diagnostic evaluation framework that moves beyond binary classification by employing prompt-level rubrics. Grounded in globally applicable AI safety policies, each rubric provides explicit criteria along two dimensions: safety, which targets concrete harms with specific exemptions to suppress false positives, and usefulness, which independently evaluates instruction fulfillment. To ensur
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.29225, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange