FB-Net: Multi-sensor fusion for object detection using front view and bird’s-eye-view features

Abstract

As a cornerstone of autonomous driving technology, environmental perception directly impacts the safety, stability, and reliability of autonomous vehicles. However, the perception systems of autonomous vehicles struggle to accurately detection occluded traffic participants. In such situations, relying on a single sensor in a single perspective alone is inadequate to meet the perception requirements of autonomous driving. Therefore, this work proposes FB-Net, a novel multimodal object detection network. Unlike traditional works, FB-Net integrates semantic and spatial information by fusing multi-sensor data from FV (Front-View) and BEV (Bird’s-Eye View) features based on self-attention mechanism. Validation on the STF dataset demonstrates that FB-Net exhibits superior performance compared to current state-of-the-art methods, with an average AP improvement of 3.72%, particularly in small object detection, where the accuracy improves by 12.32%. In addition, Experiments conducted on vehicles equipped with FB-Net further validates the model’s effectiveness and broad applicability.