Newest papers

DeclineCraft: A RAG Abstention Benchmark for Jointly Measuring Combined Retrieval and Parametric Knowledge

Atharva Thakur, Abulhair Saparov, Bruno Ribeiro · 2026 · arXiv

DOI: 10.48550/arXiv.2610.28340 · open-access PDF

Indonesian

Reliable retrieval-augmented generation (RAG) membutuhkan pengenalan ketika tidak ada dokumen yang didapat atau pengetahuan parameter yang mendukung jawaban. Kami memperkenalkan DeclineCraft, benchmark abstention dinamis yang pertanyaan yang tidak dapat dijawab menargetkan tambahan fitur Minecraft setelah corpus cut-off, dengan audit kemungkinan paparan parameter. kemudian cut-off mendukung evaluasi model masa depan. Di atas 27 LLM, kami mengukur Correct Abstention Rate (CAR; seberapa sering model mengakui pengetahuan yang tidak cukup tentang pertanyaan yang tidak dapat dijawab), Answerable Attempt Rate dan Answer Accuracy.

Machine translation (Meta M2M100); the English original follows.

Abstract

Reliable retrieval-augmented generation (RAG) requires recognizing when neither retrieved documents nor parametric knowledge supports an answer. We introduce DeclineCraft, a dynamic abstention benchmark whose unanswerable questions target Minecraft feature additions after a corpus cut-off, with audits of possible parametric exposure. Later cut-offs support evaluation of future models. Across 27 LLMs, we measure Correct Abstention Rate (CAR; how often models acknowledge insufficient knowledge on unanswerable questions), Answerable Attempt Rate and Answer Accuracy. We find that (1) single-shot r

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange