Pavement-DETR: A High-Precision Real-Time Detection Transformer for Pavement Defect Detection

Abstract

The accurate detection of road defects is crucial for enhancing the safety and efficiency of road maintenance. This study focuses on six common types of pavement defects: transverse cracks, longitudinal cracks, alligator cracking, oblique cracks, potholes, and repair marks. In real-world scenarios, key challenges include effectively distinguishing between the foreground and background, as well as accurately identifying small-sized (e.g., fine cracks, dense alligator cracking, and clustered potholes) and overlapping defects (e.g., intersecting cracks or clustered damage areas where multiple defects appear close together). To address these issues, this paper proposes a Pavement-DETR model based on the Real-Time Detection Transformer (RT-DETR), aiming to optimize the overall accuracy of defect detection. To achieve this goal, three main improvements are proposed: (1) the introduction of the Channel-Spatial Shuffle (CSS) attention mechanism in the third (S3) and fourth (S4) stages of the ResNet backbone, which correspond to mid-level and high-level feature layers, enabling the model to focus more precisely on road defect features; (2) the adoption of the Conv3XC structure for feature fusion enhances the model’s ability to differentiate between the foreground and background, which is achieved through multi-level convolutions, channel expansion, and skip connections, which also contribute to improved gradient flow and training stability; (3) the proposal of a loss function combining Powerful-IoU v2 (PIoU v2) and Normalized Wasserstein Distance (NWD) weighted averaging, where PIoU v2 focuses on optimizing overlapping regions, and NWD targets small object optimization. The combined loss function enables comprehensive optimization of the bounding boxes, improving the model’s accuracy and convergence speed. Experimental results show that on the UAV-PDD2023 dataset, Pavement-DETR improves the mean average precision (mAP) by 7.7% at IoU = 0.5, increases mAP by 8.9% at IoU = 0.5–0.95, and improves F1 Score by 7%. These results demonstrate that Pavement-DETR exhibits better performance in road defect detection, making it highly significant for road maintenance work.