Balanced sampling meets imbalanced datasets for SAR image classification

Abstract

Deep learning models trained on imbalanced datasets with long-tailed label distributions are biased towards the most frequently occurring classes, giving rise to a large number of false-negatives in the tail classes. In this paper, we utilize a cross-modal knowledge distillation framework with balanced sampling for learning SAR image classification from an Electro-Optical (EO) image classification model that is trained on co-registered data. Knowledge distillation from the soft outputs of the EO model aids in the effective training of the SAR model. However, a class balanced sampler adversely affects the performance of the head classes. To mitigate these negative effects, we propose Balanced Cross-KD to efficiently train the SAR model, end-to-end manner in a single stage, via a carefully crafted sampling strategy that strikes a balance between instance and class balanced sampling. Balanced Cross-KD performs training on a long-tailed EO/SAR dataset by alternating between instance and class balanced sampling at fixed intervals. Training utilizes our novel distributed entropy loss and equal diversity loss to encourage compact yet diverse prediction probabilities. Additionally, pretraining the SAR network on another SAR dataset is considered to obtain improved features and an ablation study further demonstrates the utility of each component of our model. Our Balanced Cross-KD model improves performance across the tail classes and increases overall mean per class accuracy, while minimally compromising performance for the head classes on a registered EO/SAR dataset.