Newest papers

Loopholing All the Way: Language Modeling with pure Self-Conditioning

Adrien Deschenaux, Nadezhda Ilieva, Justin Deschenaux, Manuel Madeira, Dorina Thanou, Aurélien Lucchi, Antonio Orvieto, Çağlar Gülçehre, Pascal Frossard B · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30068 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Los modelos de lenguaje de difusión recientes (DLMs) utilizan el autocondicionamiento (SC), o loopholing, para mejorar su rendimiento. Mientras que el SC es tradicionalmente visto como un heurístico auxiliar, su éxito empírico plantea la pregunta: ¿puede un modelo generativo continuo basarse puramente en el autocondicionamiento? Nosotros respondemos afirmativamente con LAW (Loopholing All the Way), que propaga el Embedding esperado bajo la predicción del denoiser a través de pasos, decodificando los tokens una vez, al final. En Sudoku, LAW supera a los modelos de difusión discretos y continuos anteriores, incluyendo Loopholing Discrete Diffusion. En Maze, LAW es el más preciso de 2 a 64 NFE. Después de entrenar en TinyGSM y

Machine translation (Meta M2M100); the English original follows.

Abstract

Recent Diffusion Language Models (DLMs) use Self-Conditioning (SC), or Loopholing, to improve their performance. While SC is traditionally seen as an auxiliary heuristic, its empirical success raises the question: can a continuous generative model be based purely on Self-Conditioning? We answer affirmatively with LAW (``Loopholing All the Way''), which propagates the Expected Embedding under the denoiser's prediction across steps, decoding the tokens once, at the end. On Sudoku, LAW outperforms previous Discrete and Continuous Diffusion Models, including Loopholing Discrete Diffusion. On Maze, LAW is the most accurate from 2 to 64 NFE. After training on TinyGSM and sampling at temperature 1 with 1024 NFE, LAW solves 42.2% of GSM8K problems against 34.5% for uniform Loopholing Discrete Diffusion. Additionally, fine-tuning a pre-trained Uniform Diffusion Model (Duo) with LAW increases its accuracy from 17.2% to 48.1%. Finally, on OpenWebText, LAW pushes the Pareto frontier and approaches Autoregressive Models (ARMs) with 64 NFE. Find our code at https://github.com/ilievanadezhda/law.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Language model, Computer science, Generative grammar, Autoregressive model, Embedding, Decoding methods

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange