Automated Detection and Classification of Pavement Distresses using 3D Pavement Surface Images and Deep Learning

Abstract

Pavement distresses lead to pavement deterioration and failure. Accurate identification and classification of distresses helps agencies evaluate the condition of their pavement infrastructure and assists in decision-making processes on pavement maintenance and rehabilitation. The state of the art is automated pavement distress detection using vision-based methods. This study implements two deep learning techniques, Faster Region-based Convolutional Neural Networks (R-CNN) and You Only Look Once (YOLO) v3, for automated distress detection and classification of high resolution (1,800 × 1,200) three-dimensional (3D) asphalt and concrete pavement images. The training and validation dataset contained 625 images that included distresses manually annotated with bounding boxes representing the location and types of distresses and 798 no-distress images. Data augmentation was performed to enable more balanced representation of class labels and prevent overfitting. YOLO and Faster R-CNN achieved 89.8% and 89.6% accuracy respectively. Precision-recall curves were used to determine the average precision (AP), which is the area under the precision-recall curve. The AP values for YOLO and Faster R-CNN were 90.2% and 89.2% respectively, indicating strong performance for both models. Receiver operating characteristic (ROC) curves were also developed to determine the area under the curve, and the resulting area under the curve values of 0.96 for YOLO and 0.95 for Faster R-CNN also indicate robust performance. Finally, the models were evaluated by developing confusion matrices comparing our proposed model with manual quality assurance and quality control (QA/QC) results performed on automated pavement data. A very high level of match to manual QA/QC, namely 97.6% for YOLO and 96.9% for Faster R-CNN, suggest the proposed methodology has potential as a replacement for manual QA/QC.