Newest papers

Large Flow Language Models

Arnav Chavan, Chanhyuk Lee, Udbhav Bamba, Jaehoon Yoo, Seunghoon Hong, Aditi Raghunathan, Jinwoo Kim · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28559 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

Modelos de linguagem de fluxo contínuo oferecem um caminho para a geração paralela com ferramentas para controle e aceleração indisponíveis para difusão discreta, mas técnicas eficazes para construí-los na escala de bilhões de parâmetros permanecem menos estabelecidas. Nós apresentamos o Lafa (Large Flow Language Model), um modelo de linguagem de fluxo rápido e de alto desempenho escalado para parâmetros de 8B de forma eficiente em dados. O Lafa é um modelo de fluxo em bloco com atenção causal ao prefixo e token bidirecional, fluxo assíncrono mais eficiente do que os modelos de difusão anteriores. Este design permite treinamento eficiente através da adaptação de modelos de linguagem de fluxo autoregressivo pré-treinado (AR) e de inferência rápida através de cache

Machine translation (Meta M2M100); the English original follows.

Abstract

Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet effective techniques for building them at the billion-parameter scale remain less established. We present Lafa (Large flow language model), a fast, high-performing flow language model scaled to 8B parameters in a data-efficient manner. Lafa is a blockwise flow model with causal attention on the prefix and bidirectional, asynchronous flow within each block. This design enables efficient training by adapting pretrained autoregressive (AR) models and fast inference via prefix caching and blockwise parallel decoding. We train Lafa at 0.5B, 7B, and 8B scales with fewer than 30B tokens, more efficient than previous diffusion models, and evaluate it on instruction following, reasoning, and coding, where the learned flow matches or surpasses the quality of AR and can be distilled into flow maps for one- and few-step generation. Notably, flow training preserves or even improves pretrained AR capabilities via a multi-token prediction effect, which we leverage in a self-speculative decoding scheme that verifies flow-based drafts autoregressively within a single model. This yields 2.54x the throughput of an AR model at comparable quality, outperforming multi-token prediction and discrete diffusion methods. Our findings show that continuous flows can serve as a scalable foundation for language modeling.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Prefix, Computer science, Language model, Scalability, Leverage (statistics), Flow (mathematics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange