Newest papers

Looped Uniform Diffusion Language Models

Saiteja Utpala, Chieh-Hsin Lai, Yuki Mitsufuji, Praneeth Vepakomma · 2026 · arXiv

DOI: 10.48550/arXiv.2610.30066 · open-access PDF

Chinese

统一传播语言模型(UDLMs)通过反复消化常规词汇代币的序列来生成文本,其中任何一个都可能被破坏并随后修订。在培训过程中,UDLMs必须从一个可能不值得信赖的背景中消化,而伪造的传播语言模型(MDLMs)总是在一个清洁的背景下条件。这个具有挑战性的培训目标使UDLMs比MDLM更为FLOP密集的训练,但它带来了更强的下游推理性能。

Machine translation (Meta M2M100); the English original follows.

Abstract

Uniform diffusion language models (UDLMs) generate text by iteratively denoising sequences of ordinary vocabulary tokens, any of which may be corrupted and subsequently revised. During training, UDLMs must denoise from a context that may itself be unreliable, whereas masked diffusion language models (MDLMs) always condition on a clean context. This challenging training objective makes UDLMs more FLOP-intensive to train than MDLMs, but it yields stronger downstream reasoning performance. Recently, selective looping, which recursively applies a few early Transformer layers within each denoising

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange