Newest papers

Attn-Forcing: Learning Full-Context Attention Proxies for Streaming Video Generation

Junyao Gao, Sibo Liu, Jiaxing Li, Yanan Sun, Hanxin Zhu, Weidong Zhang, Jun Zhang, Cairong Zhao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30036 · cited by 0 works (OpenAlex) · open-access PDF

German

Die Erhöhung der Kontextlänge der Aufmerksamkeit verbessert die Qualität der Streaming-Video-Generation erheblich, während sie verbotene Speicher- und Berechnungskosten mit zunehmendem Kontext verursacht. Bestehende Methoden versuchen, diese kontextuelle Informationen durch Aufmerksamkeitswahl oder -komprimierung zu erfassen, bleiben jedoch durch begrenzte Darstellungen begrenzt und fehlen explizite Aufmerksamkeits-Ebene-Überwachung, um historische Beiträge zu aktuellen Abfragen zu behalten. Um dies zu lösen, führen wir Attn-Forcing ein, ein neues Framework, das vollkontextuelle Aufmerksamkeitsbeiträge für die Streaming-Video-Generation unter begrenztem Gedächtnis und Berechnung bewahrt. Wir erreichen dies

Machine translation (Meta M2M100); the English original follows.

Abstract

Increasing the attention context length significantly improves streaming video generation quality, while incurring prohibitive memory and computational costs as the context grows. Existing methods attempt to capture this contextual information through attention selection or compression, but remain limited by bounded representations and lack explicit attention-level supervision to retain historical contributions to current queries. To address this, we introduce Attn-Forcing, a novel framework that preserves full-context attention contributions for streaming video generation under bounded memory and computation. We achieve this by introducing attention proxies that continuously approximate the attention responses induced by the historical context. Specifically, we implement the proxies as learnable visual latents and employ a Proxy Update Module to incorporate visual context from historical latents. Moreover, we propose an Attention Matching Loss to explicitly constrain the attention behavior of the learned proxies using full-context attention supervision. Extensive qualitative and quantitative experiments demonstrate that Attn-Forcing consistently improves streaming video generation quality and achieves state-of-the-art performance over existing baseline methods.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Context (archaeology), Proxy (statistics), Matching (statistics), Bounded function, Baseline (sea)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange