Newest papers

AEON: Unifying Video and 3D World Models

Weiqi Zhang, Wenyuan Zhang, Yu-Shen Liu, Junsheng Zhou · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28631 · cited by 0 works (OpenAlex) · open-access PDF

German

Aktuelle Weltmodelle für explizite Weltsimulation folgen typischerweise zwei unterschiedlichen Wegen: (1) Video-generative Modelle (z. B. Sora), die realistische Erscheinungen simulieren, aber die 3D-Konsistenz nicht bewahren; und (2) 3D-generative Modelle (z. B. Marble), die geometrisch konsistente Szenen erzeugen, aber oft statische, weniger realistische Beobachtungen erzeugen. Wir präsentieren AEON, ein einheitlicher Rahmen, der video-basierte und 3D-basierte Weltmodelle überbrückt und die Vorteile beider kombiniert. Der Schlüsselwissen von AEON liegt in einem generativen Modell, das innerhalb der text- oder bildbedingten Video-latent-Umgebung arbeitet, während renderbare 4D-Grafik-Primitiven

Machine translation (Meta M2M100); the English original follows.

Abstract

Current world models for explicit world simulation typically follow two distinct pathways: (1) video generative models (e.g., Sora), which simulate realistic appearances but fail to preserve 3D consistency; and (2) 3D generative models (e.g., Marble), which generate geometrically consistent scenes but often yield static, less realistic observations. We present AEON, a unified framework that bridges video-based and 3D-based world models and combines the benefits of both. The key insight of AEON lies in a generative model that operates within the text- or image-conditioned video latent space while producing renderable 4D graphics primitives---specifically, 4D Gaussian Splatting (4DGS)---as the explicit world representation. We train a 4D reconstruction model to predict both 4DGS and camera poses from monocular videos, serving as our initialized 4D decoder. Furthermore, we design a domain adapter that translates generative video latents into reconstructive 4D latents through a distillation process on the video VAE. AEON naturally supports video-based world modeling by rendering 4DGS at predicted camera trajectories, and constructs a persistent 4D environment with scene dynamics for realistic world simulation. Consequently, AEON empowers a diverse range of world modeling applications and achieves state-of-the-art performance on standard benchmarks. Project Page: https://weiqi-zhang.github.io/AEON

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Generative grammar, Rendering (computer graphics), Artificial intelligence, Generative model, Computer graphics

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange