A Vehicle–Infrastructure Cooperative Perception Network Based on Multi-Scale Dynamic Feature Fusion

Abstract

Vehicle-infrastructure cooperative perception enhances the perception capabilities of autonomous vehicles by facilitating the exchange of complementary information between vehicles and infrastructure. However, real-world environments often present challenges such as differences in sensor resolution and installation angles, which create a domain gap that complicates the integration of features from these two sources. This domain gap can hinder the overall performance of the perception system. To tackle this issue, we propose a novel vehicle–infrastructure cooperative perception network designed to effectively bridge the feature integration between vehicle and infrastructure sensors. Our approach includes a Multi-Scale Dynamic Feature Fusion Module designed to comprehensively integrate features from both vehicle and infrastructure across spatial and semantic dimensions. For feature fusion at each scale, we introduce the Multi-Source Dynamic Interaction Module (MSDI) and the Per-Point Self-Attention Module (PPSA). The MSDI dynamically adjusts the interaction between vehicle and infrastructure features based on environmental changes, generating enhanced interacting features. Subsequently, the PPSA aggregates these interacted features with the original vehicle–infrastructure features at the same spatial location. Additionally, we have constructed a real-world vehicle–infrastructure cooperative perception dataset, DZGSet, which includes multi-category annotations. Extensive experiments conducted on the DAIR-V2X and our self-collected DZGSet datasets demonstrate that our proposed method achieves Average Precision (AP) scores at IoU 0.5 of 0.780 and 0.652, and AP scores at IoU 0.7 of 0.632 and 0.493, respectively. These results indicate that our proposed method outperforms existing cooperative perception methods. Consequently, the proposed approach significantly improves the performance of cooperative perception, enabling more accurate and reliable autonomous vehicle operation.