Newest papers

Low-Bit N-Gram Embeddings for Language Models

Stephen Chung, Wenyu Du · 2026 · arXiv

DOI: 10.48550/arXiv.2610.31646 · open-access PDF

Chinese

许多最近的开放式语言模型使用n-gram嵌入式表来提高性能。这些表存储了短代币序列的学习表示,例如连续代币的对或三倍,该模型在处理过程中获取。然而,它们可能需要大量的内存:例如,DeepSeek-V4.1-Flash将196亿个参数分配给其Engram n-gram内存。因此,我们研究是否可以降低这种内存成本,同时保持表提供的性能提高。

Machine translation (Meta M2M100); the English original follows.

Abstract

Many recent open-weight language models use n-gram embedding tables to improve performance. These tables store learned representations of short token sequences, such as pairs or triples of consecutive tokens, which the model retrieves during processing. However, they can require substantial memory: DeepSeek-V4.1-Flash, for example, allocates 196 billion parameters to its Engram n-gram memory. We therefore investigate whether this memory cost can be reduced while preserving the performance improvement that the tables provide. We propose storing n-gram embeddings at low precision and systematica

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange