Real-Time Pedestrian Detection and Tracking Based on YOLOv3

Abstract

Lidar based 3D object and tracking is an essential part of autonomous driving. Point cloud data can provide the geometric information among the surrounding objects. However, due to the sparsity and unorderness of the point cloud of pedestrians and cyclists, it is hard to detect and track the pedestrians with a good performance and in real-time at the same time. In order to detect and track the pedestrians and cyclists in real time, we proposed a model, which is based on the YOLOv3 model, a real-time 3D object detector based on the point cloud. First of all, we use the multi-view (MV3D) and Complex-YOLO idea to transform the point cloud into a BEV map. After getting the BEV map, a modified YOLOv3 model is utilized to detect all the pedestrians and cyclists in it; YOLOv3 is an image-based detector, which uses a darknet-53 network and can detect the multi-class objects in a fast speed. Then, with the SORT algorithm, we associate the detections of the pedestrians and cyclists we get in consecutive frames and give them a changeless ID and a 3D detection box. To evaluate our model, we use the widely used KITTI object and tracking objects' data sets and get an average of 0.86 mAP on the BEV map. And the speed can also reach more than 18 frames per second, which can balance the inference time and disposing efficiency in a good performance.