Atharva Thakur, Abulhair Saparov, Bruno Ribeiro · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28340 · open-access PDF
신뢰할 수 있는 검색 증대 생성 (RAG)은 검색된 문서나 매개 변수 지식이 응답을 지원하지 않을 때 인식이 필요합니다. 우리는 DeclineCraft를 도입합니다. 신뢰할 수 없는 질문은 corpus cut-off 후 Minecraft 기능 추가를 목표로하며 가능한 매개 변수 노출에 대한 감사를 필요로합니다. 나중에 cut-offs는 미래 모델의 평가를 지원합니다. 27 LLM에서 우리는 Correct Abstention Rate (CAR; 얼마나 자주 모델이 대답할 수없는 질문에 대한 부적절한 지식을 인정하는지), Answerable Attempt Rate 및 Answer Accuracy를 측정합니다.
Machine translation (Meta M2M100); the English original follows.
Reliable retrieval-augmented generation (RAG) requires recognizing when neither retrieved documents nor parametric knowledge supports an answer. We introduce DeclineCraft, a dynamic abstention benchmark whose unanswerable questions target Minecraft feature additions after a corpus cut-off, with audits of possible parametric exposure. Later cut-offs support evaluation of future models. Across 27 LLMs, we measure Correct Abstention Rate (CAR; how often models acknowledge insufficient knowledge on unanswerable questions), Answerable Attempt Rate and Answer Accuracy. We find that (1) single-shot r
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28340, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange