Weakly Supervised Semantic Segmentation Using Hierarchical Multi-Image Model

Abstract

In this paper, we are going to discuss the problem of weakly supervised semantic segmentation. Semantic segmentation is the problem of clustering pixels in an image which they belong to the same semantic class. This task requires a training dataset where the label of each pixel in an image has been specified. However, due to the lack of this type of ground-truth knowledge, we choose weakly supervised learning methods which use weakly labeled dataset. In a weakly labeled dataset, each image is tagged by a set of labels which indicate the presence of those semantic labels within the image, unlike the fully labeled datasets, there is no information concerning each semantic class location. We propose a Hierarchal Multi-Image Model (HMIM) where it is a two-layer graphical model. In the first layer, superpixels of all the training images are connected via the local and global appearance similarity. In the second layer, similar object candidates over all the training images are also connected. Moreover, there are some connections between the first and the second layer which consider the consistency both locally and globally. Our proposed approach incorporates the object level information as well as local region appearance in weakly supervised semantic segmentation. To evaluate our proposed approach, it is applied to the MSRC-21 dataset and we achieve comparable results.