Newest papers

DeclineCraft: A RAG Abstention Benchmark for Jointly Measuring Combined Retrieval and Parametric Knowledge

Atharva Thakur, Abulhair Saparov, Bruno Ribeiro · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28340 · cited by 0 works (OpenAlex) · open-access PDF

Arabic

ويستخدمون DeclineCraft، وهو نموذج استبعاد ديناميكي يهدف إلى إضافات ميكروسوفت ذات الصلة بعد تقسيم كوربس، مع التحقيقات على التهديدات النموذجية المحتملة. يقدمون نموذج استبعاد ديناميكي، وهو نموذج استبعاد ديناميكي يهدف إلى إضافات ميكروسوفت ذات الصلة بعد تقسيم كوربس، مع التحقيقات على التهديدات النموذجية المحتملة. يقدمون نموذج استبعاد ديناميكي إضافي، وهو نموذج استبعاد ديناميكي يهدف إلى إضافات ميكروسوفت ذات الصلة بعد تقسيم كوربس. يجدون أن (1) نموذج استبعاد واحد يساعد بشكل كبير في تقييم

Machine translation (Meta M2M100); the English original follows.

Abstract

Reliable retrieval-augmented generation (RAG) requires recognizing when neither retrieved documents nor parametric knowledge supports an answer. We introduce DeclineCraft, a dynamic abstention benchmark whose unanswerable questions target Minecraft feature additions after a corpus cut-off, with audits of possible parametric exposure. Later cut-offs support evaluation of future models. Across 27 LLMs, we measure Correct Abstention Rate (CAR; how often models acknowledge insufficient knowledge on unanswerable questions), Answerable Attempt Rate and Answer Accuracy. We find that (1) single-shot retrieval significantly raises CAR in 38 of 40 configurations (model and thinking setting) versus no retrieval. (2) Iterative RAG, in which the model self-assesses its context and can request up to two further retrieval rounds, significantly lowers CAR versus single-shot RAG in 30 of 40 configurations (mean 46.4%), with little change in Answer Accuracy (68.5% to 69.3%). Adding query rewriting and reranking to hybrid (dense & sparse) retrieval lowers CAR for all 12 models tested. (3) Larger models answer more accurately (Spearman $ρ= +0.60$ with total parameters), but neither model size nor release date shows a significant association with CAR. (4) Without retrieval, enabling thinking on the same weights lowers CAR significantly for 8 of 13 models (by 14% points on average), and the responses shift from acknowledged uncertainty to premise denial.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Context (archaeology), Benchmark (surveying), Parametric statistics, Feature (linguistics), Premise

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange