Newest papers

RealMe: Immersive Volumetric Video from Minimal Captures

Chenxu Zhou, Zhiyuan Yu, Yudong Jin, Yinji ShenTu, Yu Zhang, Jiaming Sun, Xiaowei Zhou · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30023 · cited by 0 works (OpenAlex) · open-access PDF

Korean

이 논문은 소규모 및 낮은 해상도 캡처 리소스에서 몰입 데이터 볼륨 비디오를 생성하는 문제를 해결합니다. 전통적인 밀접한 멀티 비디오 재구성 방법은 고품질 볼륨 비디오를 생성 할 수 있지만 비용이 많이 드는 캡처 설정에 의존하여 실제 시나리오에서 비용이 부담스럽습니다. 최근의 접근 방식은 실험 캡처 요구 사항을 완화하기 위해 생성 모델을 사용하지만 여전히 제한된 해상도와 4D 불일치를 앓고 있습니다. 이러한 제한을 극복하기 위해 우리는 RealMe를 제안합니다. 높은 해상도 비디오 확산 모델을 중심으로 구축 된 통합 프레임 워크는 높은 해상도 (2K)로 제한된 캡처를 향상시킵니다. 4D

Machine translation (Meta M2M100); the English original follows.

Abstract

This paper addresses the problem of creating immersive volumetric videos from sparse and low-resolution capture resources. Conventional dense multi-view reconstruction methods can produce high-quality volumetric videos, but their reliance on costly capture setups makes them impractical in real-world scenarios. Recent approaches use generative models to relax capture requirements, but still suffer from limited resolution and 4D inconsistency. To overcome these constraints, we propose RealMe, a unified framework built around a high-resolution video diffusion model that enhances limited captures into high-resolution (2K), 4D-consistent videos for volumetric video reconstruction. Specifically, RealMe constructs auxiliary data from capture videos and encodes them into latents along with layout-aware positional embeddings. The video diffusion model adapts factorized spatio-temporal attention to the multi-view capture layout, denoises these latents across viewpoints and time, and decodes them into enhanced videos to refine the 4D representation. This adaptation helps the model capture temporal dynamics and multi-view relationships, leading to more 4D-consistent generation. To reduce the data gap between generated videos and real captures, we apply a latent frequency loss that provides frequency-domain supervision in latent space. Extensive experiments on the public DNA-Rendering benchmark and our self-captured RealVV dataset demonstrate that RealMe is able to create immersive volumetric videos and outperforms state-of-the-art methods in visual fidelity and 4D consistency.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Computer vision, Motion capture, Benchmark (surveying), Generative model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange