Real-time Object Detection based on Fusion of Vision and Point-cloud Interpolated LiDAR

Abstract

This paper aims to detect objects from BEV (Bird-Eye View) images by utilizing multi-source data from three-dimensional light detection and ranging (3D LiDAR) and a camera. Object detection via the camera is implemented by using a single-stage detector, YOLO-v3, and the distance of the detected objects is estimated by performing camera calibration with the 3D LiDAR. BEV images for sensor fusion are generated using LiDAR data. Sensor fusion estimates the distance of objects detected by the camera and enhances the features in BEV images using point interpolation based on point cloud data obtained by the 3D LiDAR. Object detection is performed by applying the proposed deep neural network on enhanced BEV images. Evaluation is performed using the KITTI validation dataset and a privately collected dataset, which includes images obtained on actual roads.