Attention multilayer feature fusion network for 3D object detection

Abstract

The 3D object detection based on the fusion of 3D point cloud and 2D image is becoming a hot research topic in the field of 3D scene understanding. The key to fusion research is how to effectively fuse these two modes without information loss and interference from different sensor data. To solve this problem, we propose a multi-layer feature fusion framework based on attention mechanism, which takes 3D point cloud and 2D image as inputs for 3D object detection. In order to comprehensively consider the detection of objects of different sizes, we propose a depth fusion module, which extracts local and global features based on the summing fusion of features point by point. Based on this, we propose an attention-based fusion module to effectively fuse multilayer features by estimating the importance of three-level features through attention mechanism, thus achieving adaptive fusion of multi-layer features. Our evaluation experiments were conducted on the KITTI 3D object detection dataset. The proposed AMFF-Net performs consistently well compared to other state-of-the-art methods, particularly in terms of 3D Average Precision (AP) for the "Car" category. It also outperforms most fusion methods in detecting small targets like pedestrians in complex 3D environments. These results have been validated on the KITTI online testing dataset as well.