Newest papers

Drive2Gauss: FeedForward 4D Gaussian Reconstruction from Multimodal Generative Driving Videos

Tong Zhao, Zhiyuan Han, Lin Chen, Bowen Xie, Yunxi Qiao, Bohan Li, Weiqing Xiao, Gen Li, Shu Han, Xuyang Dai, Cheng Bi, Hao Zhao, Chaojian Li · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26585 · cited by 0 works (OpenAlex) · open-access PDF

Korean

재사용 가능한 고정 운전 환경을 생성하기 위해서는 다양한 관점에서 렌더링을 지원하는 제어 가능한 미래 관찰과 명시적 표현이 모두 필요하지만 기존 방법은 여전히 각 시퀀스에 대한 최적화를 통해 생성된 RGB 비디오에서 명시적인 장면을 복구합니다. 이러한 제한을 극복하기 위해 우리는 Drive2Gauss를 소개합니다. 공동 생성된 RGB, 메트릭 깊이 및 동적 흐름을 동적 4D Gaussian 장면으로 변환하는 feedforward 프레임 워크입니다. Multimodal Video Generator는 참조 multiview 이미지와 미래 제어 신호에서 세 가지 모형을 예측합니다. 또한, 우리는 각 장면에 대한 단일 passforward 최적화를 사용하지 않고 Gauss Query Decoder를 소개합니다. https://Gaub

Machine translation (Meta M2M100); the English original follows.

Abstract

Generating reusable driving environments requires both controllable future observations and an explicit representation that supports rendering from different viewpoints, yet existing methods still recover explicit scenes from generated RGB videos through optimization for each sequence. To overcome these limitations, we present Drive2Gauss, a feedforward framework that converts jointly generated RGB, metric depth, and dynamic flow into dynamic 4D Gaussian scenes. A Multimodal Video Generator predicts the three modalities from reference multiview images and future control signals. Furthermore, we introduce a Gaussian Query Decoder that uses depth to initialize sparse 3D queries and flow to associate the queries across time, and integrates appearance information through spatiotemporal feature aggregation and iterative query refinement. The reconstruction stage operates in a single feedforward pass without optimization for each scene. Extensive results on nuScenes demonstrate that our approach attains an FID of 6.38 and an FVD of 12.43 for future RGB generation, a PSNR of 29.91 dB when reconstructing its generated observations, and supports both direct rendering under lateral camera shifts and downstream video fixing. Code and more visualizations are available at https://zttt04. github.io/Drive2Gauss_page/.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Rendering (computer graphics), Computer vision, Feed forward, RGB color model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange