Slimming Convolutional Neural Network Based on Attention Mechanism for Pavement Crack Detection

Abstract

In recent years, convolutional neural networks (CNNs) have achieved excellent performance in pavement crack segmentation tasks. However, CNNs usually need high computational cost and large storage. As a result, CNN slimming is important for embedded systems with limited resources. Inspired by the channel attention techniques in CNNs, we propose an effective method to slim CNNs based on attention mechanism. The proposed method first removes the unimportant filters and their connecting feature maps according to their attention scales. The retraining process is then performed on the slimmed model to regain performance. The proposed pruning method is easily embedded in common convolutional layers without any dedicated software libraries. Comprehensive experiments show that the proposed method can prune over 40% parameters of CNNs applied on pavement crack segmentation tasks while the deterioration in performance is slight compared with the original UNet architecture.