Dual Attention Equivariant Network for Weakly Supervised Semantic Segmentation

Abstract

Image-level weakly supervised semantic segmentation is a challenging problem in computer vision and has gained a lot of attention in recent years. Most existing models utilize class activation mapping (CAM) to generate initial pseudo-labels for each image pixel. However, CAM usually focuses only on the most discriminating regions of target objects and treats each channel feature map independently, which may overlook some important regions due to the lack of accurate pixel-level labels, leading to the underactivation of the target objects. In this paper, we propose a dual attention equivariant network (DAEN) model to address this problem by considering both channel and spatial information of different feature maps. Specifically, we first design a channel–spatial attention module (CSM) for DAEN to extract accurately features of target objects by considering the correlation among feature maps in different channels, and then integrate the CSM with equivariant regularization and pixel-correlation modules to achieve more accurate and effective pixel-level semantic segmentation. Extensive experimental results show that the DAEN model achieved 2.1% and 1.3% higher mIoU scores than the existing weakly supervised semantic segmentation models on the PASCAL VOC 2012 and LUAD-HistoSeg datasets, respectively, validating the effectiveness and efficiency of the DAEN model.