S⁶: Semi-Supervised Self-Supervised Semantic Segmentation

Abstract

Semi-supervised learning provides a means to leverage unlabeled data when labels are expensive to obtain. In this work, we propose a constrained framework that better learns from unlabeled data. The proposed algorithm adds a self-supervised task, image reconstruction, to the target segmentation task. The extra reconstruction task improves the model's geometric reasoning about different textures in an image. It happens that this improvement is transferable from reconstruction to segmentation since they both share some common parts of the architecture. Such extra task allows the trained model to have richer representations and better geometric understanding. Our results show that the proposed constrained framework achieves an improvement in mean Intersection over Union by 18% over unconstrained one, using only 2% of labeled examples. The performance gain and reduction in amounts of labeled data is crucial for applications in which obtaining labels is expensive and labor intensive such as Biomedical Imaging and Seismic Interpretation.