Newest papers

Drive2Gauss: FeedForward 4D Gaussian Reconstruction from Multimodal Generative Driving Videos

Tong Zhao, Zhiyuan Han, Lin Chen, Bowen Xie, Yunxi Qiao, Bohan Li, Weiqing Xiao, Gen Li, Shu Han, Xuyang Dai, Cheng Bi, Hao Zhao, Chaojian Li · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26585 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

创建可重复使用的驾驶环境需要可控制的未来观察和支持从不同的视点进行渲染的明确表示,但现有的方法仍然可以通过每个序列的优化来恢复生成的RGB视频中的明确场景。 为了克服这些局限性,我们提出了Drive2Gauss,一个可转换的 feedforward框架,该框架将联合生成的RGB、计量深度和动态流量转换成动态4D Gaussian场景。 多模式视频生成器预测了从参考多重视图和未来控制信号的三个模式。 此外,我们引入了一种Gaussian Query Decoder,该框架使用depthalase来启动分散的3D查询,并将跨时间查询联系起来,并通过空间时机功能聚合和迭代式

Machine translation (Meta M2M100); the English original follows.

Abstract

Generating reusable driving environments requires both controllable future observations and an explicit representation that supports rendering from different viewpoints, yet existing methods still recover explicit scenes from generated RGB videos through optimization for each sequence. To overcome these limitations, we present Drive2Gauss, a feedforward framework that converts jointly generated RGB, metric depth, and dynamic flow into dynamic 4D Gaussian scenes. A Multimodal Video Generator predicts the three modalities from reference multiview images and future control signals. Furthermore, we introduce a Gaussian Query Decoder that uses depth to initialize sparse 3D queries and flow to associate the queries across time, and integrates appearance information through spatiotemporal feature aggregation and iterative query refinement. The reconstruction stage operates in a single feedforward pass without optimization for each scene. Extensive results on nuScenes demonstrate that our approach attains an FID of 6.38 and an FVD of 12.43 for future RGB generation, a PSNR of 29.91 dB when reconstructing its generated observations, and supports both direct rendering under lateral camera shifts and downstream video fixing. Code and more visualizations are available at https://zttt04. github.io/Drive2Gauss_page/.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Rendering (computer graphics), Computer vision, Feed forward, RGB color model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange