Edge-guided nonlinear dynamic convolution network for lightweight semantic segmentation

Abstract

As the demand for autonomous driving and robot vision arises, semantic segmentation has developed rapidly. Semantic segmentation provides the means for machines to understand the environment. However, current methods face a balance issue between segmentation quality and computational resources. Our proposed edge-guided non-linear dynamic convolutional network (ENNet) achieves real-time and accurate semantic segmentation. The core of our network is a novel non-linear, dynamic combination (Nd-conv) that allows for a non-linear, dynamic combination (Nd-conv) of multiple convolution weights, improving the convolution's encoding ability while maintaining a low computational burden. The Non-linear Dynamic Convolutional Modules (NDCM) is introduced to enhance the segmentation accuracy of the network while maintaining efficiency. The Multi-stage Feature Fusion Module (MFFM) is also introduced to fuse low-level details and high-level semantic information to improve the segmentation accuracy of the network. Experiments conducted on a 1080Ti GPU show that our model achieves 74.6 mIOU on the Cityscapes dataset while being able to perform inference at 202 FPS. Our network design has achieved an optimal balance between speed and accuracy.