Newest papers

RealMe: Immersive Volumetric Video from Minimal Captures

Chenxu Zhou, Zhiyuan Yu, Yudong Jin, Yinji ShenTu, Yu Zhang, Jiaming Sun, Xiaowei Zhou · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30023 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Extistence aborda el problema de crear vídeos volumétricos inmersivos de datos de recursos de captura escasos y de baja resolución. Los métodos convencionales de reconstrucción multi-visual densa pueden producir vídeos volumétricos de alta calidad, pero su dependencia de configuraciones de captura costosas los hace impracticables en escenarios del mundo real. Los enfoques recientes utilizan modelos generativos para relajar los requisitos de captura experimental, pero todavía sufren de resolución limitada y inconsistencia 4D. Para superar estas restricciones, proponemos RealMe, un marco unificado de espacio latente construido en torno a un modelo de difusión de vídeo de alta resolución que mejora las capturas limitadas a alta resolución (2K), 4D-consistentes videos para la reconstrucción de vídeo volumétrico. En concreto, RealMe construye datos auxili

Machine translation (Meta M2M100); the English original follows.

Abstract

This paper addresses the problem of creating immersive volumetric videos from sparse and low-resolution capture resources. Conventional dense multi-view reconstruction methods can produce high-quality volumetric videos, but their reliance on costly capture setups makes them impractical in real-world scenarios. Recent approaches use generative models to relax capture requirements, but still suffer from limited resolution and 4D inconsistency. To overcome these constraints, we propose RealMe, a unified framework built around a high-resolution video diffusion model that enhances limited captures into high-resolution (2K), 4D-consistent videos for volumetric video reconstruction. Specifically, RealMe constructs auxiliary data from capture videos and encodes them into latents along with layout-aware positional embeddings. The video diffusion model adapts factorized spatio-temporal attention to the multi-view capture layout, denoises these latents across viewpoints and time, and decodes them into enhanced videos to refine the 4D representation. This adaptation helps the model capture temporal dynamics and multi-view relationships, leading to more 4D-consistent generation. To reduce the data gap between generated videos and real captures, we apply a latent frequency loss that provides frequency-domain supervision in latent space. Extensive experiments on the public DNA-Rendering benchmark and our self-captured RealVV dataset demonstrate that RealMe is able to create immersive volumetric videos and outperforms state-of-the-art methods in visual fidelity and 4D consistency.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Computer vision, Motion capture, Benchmark (surveying), Generative model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange