HDPNERF: Hybrid Depth Priors for Neural Radiance Fields from Sparse Input Views

Abstract

Neural Radiance Field (NeRF) shows a high prospect in the task of novel view synthesis. However, performance degrades drastically under limited input views since NeRF heavily relies on a large number of images to fit the geometry in scenes. Recent efforts focus on introducing extra constraints to improve rendering results, but problems still exist due to the lack of enough geometric information about scenes. In this work, we present a novel combination of learning-based Multi-View Stereo (MVS) and Monocular Depth Estimation (MDE) depths for high-quality geometric priors. We retain view-consistent values in MVS depths by consistency filtration. Then we introduce an MVS-guided patch-wise transformation strategy for monocular depths to resolve ambiguities and match the scale of MVS depths. Finally, we complete filtered MVS depths with transformed monocular depths to generate view-consistent, complete, and smooth geometric priors. Experiments show that our approach achieves state-of-the-art results.