Newest papers

When Copying Is Hard: Copy-Constrained Decoding for Exact Span Reproduction

Jinghui Zhang, Lang Gao, Zongfang Liu, Ruihong Zeng, Zirui Song, Rui Yan, Kentaro Inui, Xiuying Chen · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28393 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

即使目标字符串在背景中存在,模型往往仍然在范围内运行,但无法正确停止,揭示源地化和边界控制的局限性。这种能力在长的背景下具有广泛的实用性。例如,代码生成往往需要嵌入敏感字符串,如API密钥,正如它们在背景中出现的那样。在这项工作中,我们构建了一个基准来测试合同,科学文献,代码和随机序列之间的准确长度复制能力,并发现目前的模型仍然具有令人惊讶的频繁错误。为了解决这个问题,我们提出了CopyGen,一个轻量级的复制限制插件,用于冻结的LLM,这意味着从可预测的下一个任务的副产品复制到可控制的过程。我们的关键想法是将某些东西分解成

Machine translation (Meta M2M100); the English original follows.

Abstract

Precisely reproducing a target span verbatim within a long context is a fundamental capability test for LLMs: even when the target string is present in context, models often still drift within the span or fail to stop correctly, revealing limitations in source grounding and boundary control. This capability has broad practical relevance. For instance, code generation often requires embedding sensitive strings such as API keys exactly as they appear in context. In this work, we construct a benchmark to test exact span reproduction ability across contracts, scientific literature, code, and random sequences, and find that current models still make surprisingly frequent errors. To address this, we propose CopyGen, a lightweight copy-constrained plugin for frozen LLMs that turns copying from a byproduct of next-token prediction into a controlled process. Our key idea is to decompose copying into three distinct decisions: when to copy, how far to copy, and when to stop. Concretely, CopyGen first determines whether to enter copy mode, then predicts a tentative copy length, and finally verifies the generated span token by token. During verification, we adaptively adjust the logits of copy continuation tokens to incentivize or suppress continuation, and emit only the longest valid prefix. Across three LLM backbones and four benchmark tasks, CopyGen improves average exact match by 35% relative without model finetuning, and integrates seamlessly with SFT to further improve it by 4%, while substantially accelerating generation. These results suggest that reliable exact copying is not something LLMs obtain for free, but a decoding-time control problem that requires explicit modeling. Code is available at https://github.com/mbzuai-nlp/CopyGen.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Copying, Computer science, Benchmark (surveying), Context (archaeology), Decoding methods, String (physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange