Newest papers

Large Flow Language Models

Arnav Chavan, Chanhyuk Lee, Udbhav Bamba, Jaehoon Yoo, Seunghoon Hong, Aditi Raghunathan, Jinwoo Kim · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28559 · cited by 0 works (OpenAlex) · open-access PDF

German

Kontinuierliche Flusssprachenmodelle bieten einen Weg zur Parallelgeneration mit Tools zur Steuerung und Beschleunigung, die für diskrete Diffusion nicht verfügbar sind, aber wirksame Techniken, um sie auf der Milliarden-Parameter-Skala aufzubauen, bleiben weniger etabliert. Wir präsentieren Lafa (Großflusssprachenmodell), ein schnelles, leistungsstarkes Flusssprachenmodell, das auf 8B-Parameter in einer dateneffizienten Weise skaliert wird. Lafa ist ein Blockflussmodell mit kausaler Aufmerksamkeit auf das Präfix und bidirektionalen, asynchronen Fluss innerhalb jedes Blocks. Dieses Design ermöglicht eine effiziente Ausbildung durch Anpassung von prätrainierten autoregressiven (AR) Sprachenmod

Machine translation (Meta M2M100); the English original follows.

Abstract

Continuous flow language models offer a path to parallel generation with tools for control and acceleration unavailable to discrete diffusion, yet effective techniques for building them at the billion-parameter scale remain less established. We present Lafa (Large flow language model), a fast, high-performing flow language model scaled to 8B parameters in a data-efficient manner. Lafa is a blockwise flow model with causal attention on the prefix and bidirectional, asynchronous flow within each block. This design enables efficient training by adapting pretrained autoregressive (AR) models and fast inference via prefix caching and blockwise parallel decoding. We train Lafa at 0.5B, 7B, and 8B scales with fewer than 30B tokens, more efficient than previous diffusion models, and evaluate it on instruction following, reasoning, and coding, where the learned flow matches or surpasses the quality of AR and can be distilled into flow maps for one- and few-step generation. Notably, flow training preserves or even improves pretrained AR capabilities via a multi-token prediction effect, which we leverage in a self-speculative decoding scheme that verifies flow-based drafts autoregressively within a single model. This yields 2.54x the throughput of an AR model at comparable quality, outperforming multi-token prediction and discrete diffusion methods. Our findings show that continuous flows can serve as a scalable foundation for language modeling.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Prefix, Computer science, Language model, Scalability, Leverage (statistics), Flow (mathematics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange