Newest papers

JointLight: Human-Centric Video Relighting with Asynchronous RGB-Irradiance Joint Modeling

Henglei Lv, Bailin Deng, Xiaoqiang Liu, Yichen Wang, Haoxian Zhang, Pengfei Wan, Lin Gao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29780 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

人間中心のシーンにおける光の輸送の理解とモデリングは長期的な課題です。拡散モデルの最近の進歩はリフレッシュの研究を促進しているが、正確な照明制御は依然として遅れている、主に、多様で高品質で正確にラベルアップされたビデオリフレッシュデータセットの欠如のために。既存のエンド-to-エンドリフレッシュのアプローチは、神経ネットワークを通じて暗示的に光の輸送をモデル化し、コントロールを大幅に複雑にします。対照的に、いくつかの作業は、光の輸送のための明確なブループリントとして、トラッキングされた放射線のヒントを参照します。しかし、このような構築された放射線のヒントは、素材を生成

Machine translation (Meta M2M100); the English original follows.

Abstract

Understanding and modeling light transport in human-centric scenes is a longstanding challenge. While recent progress in diffusion models has boosted research in relighting, precise illumination control still lags, primarily due to the scarcity of diverse, high-quality, precisely labeled video relighting datasets. Existing end-to-end relighting approaches model light transport implicitly through neural networks, which substantially complicates controllability. In contrast, some works resort to path-traced irradiance cues as an explicit blueprint for light transport. However, such constructed irradiance cues are prone to artifacts and hinder generative quality when directly used as control signals. With the key observation that noise corruption removes high-frequency artifacts and preserves low-frequency light transport, we introduce an asynchronous joint-modeling paradigm that facilitates fine-grained illumination control. During two-stage training, we force the model to jointly denoise RGB and ground-truth irradiance frames with separate timesteps scheduling. A domain matching strategy is applied to the second stage to bridge the domain gap between irradiance cues and ground-truth irradiance maps. At inference time, we introduce an asynchronous hijacked generation pipeline that facilitates flexible illumination control. For training data, we develop a multi-illumination high-quality synthetic video dataset with paired irradiance maps using a powerful game engine. Trained exclusively on this dataset, our model exhibits robust generalization to real- world scenarios and outperforms prior approaches in both lighting fidelity and generative quality. Project page: http://www.geometrylearning.com/JointLight/

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Computer vision, Upsampling, RGB color model, Pipeline (software)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange