SemiBaCon: Semi-Supervised Balanced Contrastive Learning for Multimodal Remote Sensing Image Classification

Abstract

The limited availability of annotated training data significantly constrains the classification accuracy of hyperspectral image (HSI) and LiDAR fusion approaches. Although contrastive learning has emerged as a potential solution, current implementations frequently neglect the critical class imbalance issues during unlabeled sample selection. To address the issue, we introduce a novel semi-supervised balanced contrastive learning (SemiBaCon) framework for multi-modal remote sensing image classification. First, we propose a superpixel-based balanced sampling (SPBS) mechanism that fundamentally addresses class imbalance through intelligent pseudo-label generation. By segmenting the HSI data into homogeneous superpixels and implementing intra-region label propagation, the method ensures statistically balanced pseudo-label selection across categories, effectively overcoming the bias introduced by conventional random sampling strategies. Second, our architecture integrates a dual-stream encoder combining convolutional neural networks (CNNs) with Transformers, enabling hierarchical feature extraction from spectral-spatial characteristics of HSI and elevation patterns of LiDAR. This design facilitates the construction of multi-modal positive sample pairs, achieving enhanced representation learning through inter-modal consistency constraints. Third, we develop a pseudo-label guided contrastive learning (PLCL) paradigm that synergistically combines pseudo-label confidence with feature similarity metrics, which effectively reduces intra-class variance and improves decision boundaries in the latent space. Comprehensive evaluations on three benchmark datasets demonstrate the framework’s superior performance compared to the state-of-the-art methods.