Sparse Pseudo-LiDAR Depth Assisted Monocular Depth Estimation

Abstract

Monocular depth estimation has attracted extensive attention and made great progress in recent years. However, the performance still lags far behind LiDAR-based depth completion algorithms. This is because the completion algorithms not only utilize the RGB image, but also have the prior of sparse depth collected by LiDAR. To reduce this performance gap, we propose a novel initiative that incorporates the concept of pseudo-LiDAR into depth estimation. The pseudo-LiDAR depends only on the camera and thus achieves a lower cost than LiDAR. To emulate the scan pattern of LiDAR, geometric sampling and appearance sampling are proposed. The former measures the vertical and horizontal azimuths of 3D scene points to establish the geometric correlation. The latter helps determine which "pseudo-LiDAR rays" return an answer and which do not. Then, we build a sparse pseudo-LiDAR-based depth estimation framework. Extensive experiments show that the proposed method surpasses previous state-of-the-art competitors on the KITTI, NYU-Depth-v2 and SUN RGB-D datasets.