Robust pseudo-label selection for holistic semi-supervised learning

Abstract

Semi-supervised learning (SSL) is a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. Although it has been reported that SSL methods achieve significant performance on multiple benchmark datasets, they still have critical limitations when applied to real-world tasks, such as being difficult to determine the quality of pseudo-labels, being sensitive to hyper-parameter choices, lacking theoretical guarantee. To address these issues, we propose a new holistic SSL approach with robust pseudo-label selection. Specifically, our proposal selects pseudo-labels adaptively based on the disagreement of model predictions without pre-defined hyper-parameters. Theoretically, we prove that the classification error decreases with the training iterations. Experimentally, we achieve state-of-the-art performance by a large margin across various datasets. For example, compared with the SOTA SSL algorithm FixMatch, we reduce the error by 11.8% on the CIFAR-10 dataset and 18.8% on the more difficult STL-10 dataset.