Lightweight Semantic Segmentation Network Leveraging Class-Aware Contextual Information

Abstract

Balancing model size, segmentation accuracy, and inference speed is a key challenge in image semantic segmentation. This paper introduces a novel lightweight semantic segmentation network, CACNet (Class-Aware Context Network), featuring the innovative Class-Aware Context Enhancement Module (CACEM). CACEM is designed to explicitly intertwine category and context information, addressing the shortcomings of traditional convolutional networks in capturing and encoding inter-category relationships. It operates by normalizing pixel probability distributions via softmax, mapping pixels to categories, and generating new feature maps that accurately encapsulate these relationships. Additionally, the network utilizes multi-scale context information and employs dilated convolutions, followed by upsampling to blend this context with single-channel category information. This process, enhanced by Fourier adaptive attention mechanisms, allows CACNet to capture intricate feature structures and manipulate features in the frequency domain for improved segmentation accuracy. On the Cityscapes and CamVid datasets, CACNet demonstrates competitive accuracies of 70.8 and 74.6 respectively, with a compact model size of 0.52M and an inference speed over 58FPS on GTX 2080Ti GPU platform. This blend of compactness, speed, and accuracy positions CACNet as an efficient choice in resource-constrained environments.