Newest papers

DeclineCraft: A RAG Abstention Benchmark for Jointly Measuring Combined Retrieval and Parametric Knowledge

Atharva Thakur, Abulhair Saparov, Bruno Ribeiro · 2026 · arXiv

DOI: 10.48550/arXiv.2610.28340 · open-access PDF

Chinese

可靠的检索增量生成(RAG)需要识别,当既没有检索的文档,也没有参数知识支持答案时。我们引入了DeclineCraft,一个动态弃权基准,其未回答的问题在体裁后瞄准Minecraft功能添加,并对可能的参数暴露进行审计。后来的断片支持对未来的模型的评估。在27个LLM中,我们测量了正确弃权率(CAR;模型在无法回答的问题上承认不足知识的频率),答案尝试率和答案准确性。

Machine translation (Meta M2M100); the English original follows.

Abstract

Reliable retrieval-augmented generation (RAG) requires recognizing when neither retrieved documents nor parametric knowledge supports an answer. We introduce DeclineCraft, a dynamic abstention benchmark whose unanswerable questions target Minecraft feature additions after a corpus cut-off, with audits of possible parametric exposure. Later cut-offs support evaluation of future models. Across 27 LLMs, we measure Correct Abstention Rate (CAR; how often models acknowledge insufficient knowledge on unanswerable questions), Answerable Attempt Rate and Answer Accuracy. We find that (1) single-shot r

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange