Newest papers

Reading, Writing, Sharing, and Interacting with the World through Video

Shengqiong Wu, Meng Luo, Bobo Li, Ziwei Liu, Shuicheng Yan, Ming-Hsuan Yang, Philip H. S. Torr, Hao Fei · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31668 · cited by 0 works (OpenAlex) · open-access PDF

French

Les modèles de base vidéo ont progressé rapidement dans la compréhension et la génération de séquences visuelles, tandis que les modèles du monde vidéo cherchent à capturer la dynamique sous-jacente du monde qu'ils représentent. Malgré leurs objectifs différents, les deux concernent la relation entre les observations vidéo et un monde en évolution. Cette étude adopte la modélisation du monde comme un objectif unifiant pour examiner ces développements. En regardant une vidéo observée comme une projection partielle, perteuse et temporellement ordonnée d'un monde temporel latent, nous organisons la littérature autour de quatre opérations complémentaires: la lecture induit les propriétés du monde de la vidéo observée; l'écriture synthétise la vidéo à partir de conditions du monde spécifiées ou ap

Machine translation (Meta M2M100); the English original follows.

Abstract

Video foundation models have advanced rapidly in understanding and generating visual sequences, while video world models seek to capture the underlying dynamics of the world they depict. Despite their different objectives, both concern the relationship between video observations and an evolving world. This survey adopts world modeling as a unifying lens for examining these developments. Viewing an observed video as a partial, lossy, and temporally ordered projection of a latent temporal world, we organize the literature around four complementary operations: reading infers world properties from observed video; writing synthesizes video from specified or learned world conditions; sharing integrates understanding and generation through common modeling mechanisms; and interacting predicts and updates world states under actions and evolving observations. This perspective distinguishes temporal coherence in observed or generated sequences from consistency in the underlying world state. Across tasks, representations, models, datasets, and benchmarks, we trace a shift in emphasis from clip-level recognition and perceptual fidelity toward persistent state, controllable evolution, physical and causal validity, and evaluation grounded in interaction and downstream decisions, and we distill the open challenges that follow.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Consistency (knowledge bases), TRACE (psycholinguistics), Coherence (philosophical gambling strategy), Perspective (graphical), Fidelity

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange