Newest papers

Rethinking Video Autoencoders for Interactive World Models

Adrien Ramanana Rahary, Yutong Bai, Nicolas Dufour, Patrick Pérez, David Picard · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30129 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

他们作为模拟器,在那里代理人训练,计划或被评估,并且人类可以互动地控制。他们在移动视频自动编码器的隐藏空间中生成,其压缩设置了每个框架的成本和每个行动等待的时间。 然而,世界模型经常重复使用文本到视频生成器的自动编码器,用于离线生产,没有有效的代理时间等待,而没有他们的培训数据和代码,不能在另一个压缩时重新训练。 因此,我们问如何,在哪里和多少一个世界模型应该压缩。 我们建立CADRE,一个因果扩散表示自动编码器的开放家庭,涵盖了多个空间和时间压缩。 它的成员只在一个小瓶子上不同于冷冻的特征和运动代码,并且在另一个或几个步骤内共享自发

Machine translation (Meta M2M100); the English original follows.

Abstract

Video world models predict future frames from past frames and actions. They serve as simulators in which agents train, plan or are evaluated, and that humans can control interactively. They generate in the latent space of a video autoencoder, whose compression sets their cost per frame and how long each action waits to take effect. Yet world models often reuse the autoencoders of text-to-video generators, built for offline generation with no agent waiting and, without their training data and code, not retrainable at another compression. We therefore ask how, where and how much a world model should compress. We build CADRE, an open family of causal diffusion representation autoencoders spanning multiple spatio-temporal compressions. Its members differ only in a small bottleneck on frozen features and share a diffusion decoder self-distilled to one or a few steps. We train the same world model on each autoencoder in two domains, manipulation and navigation, with two environments each. CADRE forms most of the latency-quality frontier, especially with a moving camera. Our study also leads to four findings. At strong compression, a diffusion decoder is sharper than perceptual regression. In robot manipulation, temporal compression is more effective in the autoencoder than in the world model. Manipulation and navigation call for opposite compressions: temporal compression costs little when the camera is still, and spatial compression when it moves. Surprisingly, with a still camera, coarser spatial tokens hurt motion far more than image quality. Code, models and a live demo are at https://kyutai.org/blog/2026-10-21-cadre

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Autoencoder, Artificial intelligence, Data compression, Frame (networking), Bottleneck

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange