Joint Image Restoration For Weakly Supervised Semantic Segmentation

Abstract

Weakly Supervised Semantic Segmentation (WSSS) has received extensive attention in recent years as it achieves pixel-level segmentation by only image-level labels. However, existing WSSS methods commonly extract Class Activation Maps (CAM) from a classification network as the initial localization cues, which is always narrow and fragmentary. We interpret the issue as the fact that incomplete supervision of the classification task introduces implicit disturbance to CAM. We turn to Joint Image Restoration and propose Restored Class Activation Mapping (RCAM) to eliminate disturbance during CAM generation. RCAM consists of a conventional CAM generation network and a proposed restoration network. For the restoration network, we set up Guidance and Decoder branches for feature extraction, and introduce Pixel-Adaptive Convolution for feature decoding. To train RCAM, we extract refined pixel-level supervision from CAM by applying condition random filed, termed Joint Mask. Experiments on PASCAL VOC 2012 and MS COCO 2014 show that our method achieves a great improvement for CAM and is competitive with existing state-of-the-art methods.