Relightable Neural Radiance Fields for Novel View Synthesis

Abstract

This paper describes relighting neural radiance fields for novel view synthesis. View synthesis is the problem of using input images with corresponding camera angles to produce a photorealistic 3D model of an environment and its objects. Neural radiance fields (NeRFs) were created as a solution to view synthesis. Neural radiance field models work well for generating realistic 3D models from 2D image inputs; how-ever, they do not support changing the lighting or placing the objects from the input images into different environments. The problem comes from the fact that NeRFs rely on a neural network that is essentially overfitted to the original environment used in the training. This means an object in a given scene cannot be placed into a different scene using the NeRF neural network model. A new model, relightable neural radiance fields (ReNeRFs), has been proposed to combat this issue. ReNeRFs have the ability to control the lighting of an object and place it into novel environments using an image-based relighting approach.