Newest papers

Adaptive Multi-Token Prediction: How far should we go?

Emile Anand, Mohammed Rehan Parwani, Xinyuan Cao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28619 · cited by 0 works (OpenAlex) · open-access PDF

Korean

그럼에도 불구하고, 최선의 추측 지평은 맥락과 함께 변화를 완료: 더 긴 도면은 신뢰할 때 더 큰 보상을 제공하지만, 거부 후 언어 모델 디코딩을 가속화 할 수 있습니다. 여러 토큰을 작성하여 단일 타겟 모델 패스에서 확인하는 것과는 달리, MTP는 도면을 작성하기 전에 지평을 고정해야합니다. 더 긴 도면은 신뢰할 수있을 때 더 큰 보상을 제공하지만, 거부 후 거부 된 언어 모델 디코딩을 사용하여 검증 피드백에 대한 계산을 낭비 할 수 있습니다. 자극적인 도면을 작성하면 중지 규칙이 토큰에 따라 토큰을 결정할 수 있습니다. MTP는 도면을 작성하기 전에 지평을 고정해야하며, 모든 머리카락이 같은

Machine translation (Meta M2M100); the English original follows.

Abstract

Multi-token prediction (MTP) can accelerate language-model decoding by drafting several tokens and verifying them in a single target-model pass. However, the best speculation horizon changes with context: longer drafts offer greater amortization when reliable, but can waste compute on suffixes discarded after a rejection. Unlike autoregressive drafting, where a stopping rule can decide token by token, MTP must fix the horizon before drafting, as all heads fire in the same forward pass. We therefore formulate horizon selection as an online control problem, using verification feedback to learn how far the draft can be trusted in each context, and introduce Adaptive Multi-Token Prediction (AMTP): a lightweight controller that requires no retraining and leaves the exact verifier unchanged. To guide horizon selection, we characterize the expected number of tokens committed at each horizon via acceptance survival, the probability that a draft prefix is fully accepted. Before each round, AMTP predicts how far the draft is likely to survive and picks the horizon with the highest expected tokens per second, learning freely from every verification pass. Theoretically, we show that any online learner's prediction error translates into throughput regret against the best horizon for each context, and in turn into a bound on wall-clock completion time. With online gradient descent, this regret is sublinear. Across five draft-target pairings built on Llama-3.1-8B-Instruct with Medusa heads, Meta's MTP models, and Qwen3-4B-Inst-2507-MTP, evaluated on AIME 2025 and GSM8K, AMTP decodes up to 4.1x faster than standard decoding and matches or outperforms the fixed window in 8 of 10 settings, with throughput gains of up to 10%.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Regret, Security token, Horizon, Prefix, Decoding methods

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange