Newest papers

The Instruction Tax on Factual Recall: Base Models Can Beat Their Instruction-Tuned Counterparts

Christophe Khalil, Aya Mourad · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28407 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

我们向 AKBC LM-KBC 2026 共享任务介绍了我们的解决方案,在官方领导板上,我们获得了 0.6836 的分数,比起 0.2964 的基线的两倍多。这项任务需要使用语言模型创建所有可能的对象,最多有 32 亿个参数,无需进行检索或精细调节。我们表明,提示方法与模型架构一样重要:虽然指令调节模型版本通过其聊天界面实现了 0.5681,但基模型在提示时使用文本完成时达到 0.6836。因此,指令调节减少了事实回忆;我们将此效应称为指令税,这是由模型的重量而不是提示方法引起的。我们的最终解决方案相当简单:我们通过训练数据的一些示例来提示模型,生成 64-256

Machine translation (Meta M2M100); the English original follows.

Abstract

We present our solution to the AKBC LM-KBC 2026 Shared Task, where we achieved a score of 0.6836 on the official leaderboard, more than double the baseline of 0.2964. This task requires generating all possible objects for subject-relation pairs using a language model with up to 32 billion parameters, with no retrieval or fine-tuning involved. We show that the prompting methodology is as important as the model architecture: while the instruction-tuned model version achieves 0.5681 with its chat interface, the base model achieves 0.6836 when prompted using text completion. Thus, instruction tuning decreases factual recall; we refer to this effect as the instruction tax, which is caused by the model's weights rather than the prompting method. Our final solution is fairly straightforward: we prompt the base model with some examples from the training data, generate 64--256 outputs, and retain only the repeated ones. Two specialized methods are then applied for each relation: one improves the recall of large answer sets, and the other verifies a person's vitality before predicting their place of death.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Base (topology), Computer science, Recall, Task (project management), Natural language processing, Arithmetic

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange