Newest papers

Accept or Repair: Preserving Passage-Supported Answers in Contrastive Decoding

Jungseob Lee, Seungyoon Lee, Hyeonseok Moon, Seongtae Hong, Suhyune Son, Sugyeong Eo, Minhyuk Kim, Chanjun Park, Jaehyung Seo · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29727 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

大型语言模型通常会从训练中学到的知识中做出回应,即使提供的通道与知识相矛盾。 为了解决这个问题,对比解码在每个通道支持的预测和更改的提示下对比,但它也改变了对通道支持的正确答案。 我们表明,对通道进行检查的答案会消除这种对比。 基于这一观察,我们建议接受或修复(AoR),这是一个不受培训的程序。 我们的实验表明,如果通道支持它,AoR会保持采样答案。 否则,它会尝试使用对比解码进行两次替代,一旦通道通过这个检查,就会返回一个。 它会拒绝在通道不受欢迎时重复生成的答案的替代品。 我们的实验表明,A

Machine translation (Meta M2M100); the English original follows.

Abstract

Large language models often answer from knowledge learned during training, even when a provided passage contradicts that knowledge. To address this problem, contrastive decoding contrasts the predictions under the full and a modified prompt at every token, but it also alters correct answers that the passage supports. We show that checking answers against the passage removes this trade-off. Based on this observation, we propose Accept-or-Repair (AoR), a training-free procedure. AoR keeps a sampled answer if the passage supports it. Otherwise, it tries up to two replacements with contrastive decoding, returning one as soon as it passes this check. It rejects replacements that repeat the answer generated without the passage when the passage favors another answer. Our experiments show that AoR raises the score over sampling by 12.5 points on counterfactual passages and 2.3 on ordinary ones, surpassing eight contrastive and prompting baselines on both, including when each receives the same rejection rule. Replacing only initial answers that the passage does not support improves the average score of every decoder. These results suggest that models can follow passages that contradict their knowledge while preserving accuracy on passages that agree with it. Code will be released at https://github.com/js-lee-AI/AoR.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Counterfactual thinking, Decoding methods, Computer science, Code (set theory), Artificial intelligence, Linguistics

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange