Newest papers

JointLight: Human-Centric Video Relighting with Asynchronous RGB-Irradiance Joint Modeling

Henglei Lv, Bailin Deng, Xiaoqiang Liu, Yichen Wang, Haoxian Zhang, Pengfei Wan, Lin Gao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29780 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

理解和建模人为中心的数据场景中的光传输是一个长期的挑战。虽然最近的扩散模型的进步已经推动了放松的研究,精确的照明控制仍然滞后,主要是因为缺乏多样化,高质量,精确标记的视频放松数据集。现有的端到端放松方法通过神经网络暗示地模拟光传输,这在很大程度上不影响可控性。相比之下,一些工作都利用路径追踪的辐射线索作为光传输的明示蓝图。然而,这种建造的辐射线索容易产生文物和阻碍产生质量,当直接用作控制信号时。随着关键观察没有破坏发布的公共频率文物和保护我们的低频率光传输灵活性,我们引入了一种合并模拟模式,这将促进

Machine translation (Meta M2M100); the English original follows.

Abstract

Understanding and modeling light transport in human-centric scenes is a longstanding challenge. While recent progress in diffusion models has boosted research in relighting, precise illumination control still lags, primarily due to the scarcity of diverse, high-quality, precisely labeled video relighting datasets. Existing end-to-end relighting approaches model light transport implicitly through neural networks, which substantially complicates controllability. In contrast, some works resort to path-traced irradiance cues as an explicit blueprint for light transport. However, such constructed irradiance cues are prone to artifacts and hinder generative quality when directly used as control signals. With the key observation that noise corruption removes high-frequency artifacts and preserves low-frequency light transport, we introduce an asynchronous joint-modeling paradigm that facilitates fine-grained illumination control. During two-stage training, we force the model to jointly denoise RGB and ground-truth irradiance frames with separate timesteps scheduling. A domain matching strategy is applied to the second stage to bridge the domain gap between irradiance cues and ground-truth irradiance maps. At inference time, we introduce an asynchronous hijacked generation pipeline that facilitates flexible illumination control. For training data, we develop a multi-illumination high-quality synthetic video dataset with paired irradiance maps using a powerful game engine. Trained exclusively on this dataset, our model exhibits robust generalization to real- world scenarios and outperforms prior approaches in both lighting fidelity and generative quality. The code and dataset will be publicly released.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Computer vision, Upsampling, RGB color model, Pipeline (software)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange