Weakly supervised semantic segmentation using constrained multi-image model and saliency prior

Abstract

Building a graph model use the whole training set and solved by graph cut based algorithm is a common method in weak supervision semantic segmentation task, such as Multi-Image Model (MIM). It has two disadvantages: one is the parameter number of model increased rapidly with the scale growth of training set, which limited applied to large-scale data. Another is lack of use structure information in image internal. To solve above problems, we proposed a Constrained Multi-Image Model (CMIM) that training model with a part of the training data which acquired by our entropy based algorithm. It's made up of some components and each is a smaller graph. So, The CMIM can parallel or serial training and weaken the memory limit. To utilize the context information, we bring the saliency of image to unary potential in energy function. At first, we segment images to superpixels and extract the semantic texton forest (STF) feature. Then construct a conditional random fields (CRF) in the superpixel set from selected images. The data potential learned from STF featrue and saliency of superpixels. Finally, the labeling of superpixels converted to CRF optimization problem which can efficiency solved by alpha expansion algorithm. Experiments on the MSRC21 dataset show that the CMIM algorithm achieves accuracy comparable with some previous influential weakly-supervised segmentation algorithms.