Newest papers

Attn-Forcing: Learning Full-Context Attention Proxies for Streaming Video Generation

Junyao Gao, Sibo Liu, Jiaxing Li, Yanan Sun, Hanxin Zhu, Weidong Zhang, Jun Zhang, Cairong Zhao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30036 · cited by 0 works (OpenAlex) · open-access PDF

Korean

관심 컨텍스트 길이를 늘리는 것은 스트리밍 비디오 생성 품질을 크게 향상시키고, 컨텍스트가 성장함에 따라 금지적인 메모리 및 계산 비용을 부담합니다. 기존의 방법은 관심 선택 또는 압축을 통해이 컨텍스트 정보를 캡처하려고하지만, 제한된 표현에 의해 제한되어 있으며, 현재의 쿼리에 대한 역사적 기여를 보존하기 위해 명시적인 관심 수준의 감독이 부족합니다. 이를 해결하기 위해 우리는 Attn-Forcing를 도입합니다. 우리는 제한된 메모리 및 계산하에 스트리밍 비디오 생성에 대한 전체 컨텍스트의 관심 기여를 보존하는 새로운 프록시 모듈 인 Attn-Forcing을 도입합니다. 또한, 우리는 지속적으로 역사적 컨텍스트에 의해 유도 된 관심 응답을 접근하는

Machine translation (Meta M2M100); the English original follows.

Abstract

Increasing the attention context length significantly improves streaming video generation quality, while incurring prohibitive memory and computational costs as the context grows. Existing methods attempt to capture this contextual information through attention selection or compression, but remain limited by bounded representations and lack explicit attention-level supervision to retain historical contributions to current queries. To address this, we introduce Attn-Forcing, a novel framework that preserves full-context attention contributions for streaming video generation under bounded memory and computation. We achieve this by introducing attention proxies that continuously approximate the attention responses induced by the historical context. Specifically, we implement the proxies as learnable visual latents and employ a Proxy Update Module to incorporate visual context from historical latents. Moreover, we propose an Attention Matching Loss to explicitly constrain the attention behavior of the learned proxies using full-context attention supervision. Extensive qualitative and quantitative experiments demonstrate that Attn-Forcing consistently improves streaming video generation quality and achieves state-of-the-art performance over existing baseline methods.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Context (archaeology), Proxy (statistics), Matching (statistics), Bounded function, Baseline (sea)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange