Newest papers

Attn-Forcing: Learning Full-Context Attention Proxies for Streaming Video Generation

Junyao Gao, Sibo Liu, Jiaxing Li, Yanan Sun, Hanxin Zhu, Weidong Zhang, Jun Zhang, Cairong Zhao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30036 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

通过注意选择或压缩,现有的方法试图通过注意选择或压缩来捕获这种背景信息,但仍然受到限制,并且缺乏明确的注意级别监督以保持当前查询的历史贡献。为了解决这一问题,我们引入了Attn-Forcing,一种新框架,以保持在有限的记忆和计算下流媒体视频生成的全背景注意力贡献。我们通过引入持续接近历史背景引起的注意力响应的注意力代理来实现这一目标,具体地说,我们将代理作为可学习的可视化后代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代代

Machine translation (Meta M2M100); the English original follows.

Abstract

Increasing the attention context length significantly improves streaming video generation quality, while incurring prohibitive memory and computational costs as the context grows. Existing methods attempt to capture this contextual information through attention selection or compression, but remain limited by bounded representations and lack explicit attention-level supervision to retain historical contributions to current queries. To address this, we introduce Attn-Forcing, a novel framework that preserves full-context attention contributions for streaming video generation under bounded memory and computation. We achieve this by introducing attention proxies that continuously approximate the attention responses induced by the historical context. Specifically, we implement the proxies as learnable visual latents and employ a Proxy Update Module to incorporate visual context from historical latents. Moreover, we propose an Attention Matching Loss to explicitly constrain the attention behavior of the learned proxies using full-context attention supervision. Extensive qualitative and quantitative experiments demonstrate that Attn-Forcing consistently improves streaming video generation quality and achieves state-of-the-art performance over existing baseline methods.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Context (archaeology), Proxy (statistics), Matching (statistics), Bounded function, Baseline (sea)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange