Time-conditioned Illumination for Inverse Rendering of Outdoor Scenes


Xiaoxue Chen (Tsinghua University, Tsinghua University), Hao Zhao (Tsinghua University, Tsinghua University), Guyue Zhou (Tsinghua University), Ya-Qin Zhang (AIR, Tsinghua University)
The 35th British Machine Vision Conference
PDFPosterSupplementary

Abstract

Inverse rendering has been a long-standing problem in computer graphics and vision, with the objective of decomposing images into intrinsic scene properties including geometry, illumination, and material. While conventional works mainly focus on object-level or indoor scenes, addressing the inverse rendering problem for outdoor scenes poses more challenges, which arise from the complex geometry and time-variant nature of outdoor illumination due to changing sun position and atmosphere condition. In this paper, we present a novel inverse rendering framework tailored for outdoor scenes characterized by varying illumination. Specifically, we first disentangle the underlying geometry from appearance based on the neural radiance fields and incorporate monocular geometric cues to resolve the complexity of geometry. In addition, we introduce a time-dependent field to model the time-variant illumination from the sky dome and parameterize the material properties with the microfacet Bidirectional Reflectance Distribution Function (BRDF). Finally, we propose a differentiable re-rendering module that integrates all the decomposed properties to generate new renderings. Experiments demonstrate that our novel inverse rendering framework yields high-quality reconstruction of scenes' geometry, material, and illumination, and outperforms previous SOTA methods in the task of novel view synthesis for outdoor scenes. Moreover, this framework facilitates various scene editing applications including material editing, object removal, and relighting. Codes will be publicly available.

Citation

@inproceedings{Chen_2024_BMVC,
author    = {Xiaoxue Chen and Hao Zhao and Guyue Zhou and Ya-Qin Zhang},
title     = {Time-conditioned Illumination for Inverse Rendering of Outdoor Scenes},
booktitle = {35th British Machine Vision Conference 2024, {BMVC} 2024, Glasgow, UK, November 25-28, 2024},
publisher = {BMVA},
year      = {2024},
url       = {https://papers.bmvc2024.org/0697.pdf}
}


Copyright © 2024 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection