IPAB Seminar - 28/11/2022

Title: Self-supervised inverse rendering

Abstract:  Inverse rendering is the task of decomposing one or more images into geometry, illumination and reflectance such that these quantities would recreate the original image when rendered. Deep learning has shown great promise for solving components of this task in unconstrained situations. However, the challenge is a lack of ground truth labels to use for supervision. I will describe a line of work that learns to solve this problem for outdoor scenes with no ground truth. They are based on extracting a self-supervision signal from unstructured image collections alone while introducing model-based constraints to resolve ambiguities. I will describe both single image methods, that learn general principles of inverse rendering, and multi-image methods that fit to a single scene by extending Neural Radiance Fields to relightable outdoor scenes. I will describe priors that we enforce on natural illumination and results on the application of photorealistic scene relighting.