MMML: Multimanifold Metric Learning for Few-Shot Remote-Sensing Image Scene Classification

Abstract

Deep neural networks driven by large amounts of annotated samples have been successfully applied in remote sensing scene classification (RSSC). On most common public datasets, the accuracies of scene recognition tasks have been close to saturation. A learning paradigm that specifically tackles this issue has emerged, i.e., few-shot learning (FSL). FSL methods for natural image recognition are developing rapidly. Recently, these methods have been widely used in RSSC. Compared with natural images, the problems of intra-class differences and inter-class similarities in RSSC are more serious, which hinders the further development of FSL methods for RSSC. To address this issue, we propose a multi-manifold metric learning framework for RSSC with FSL. Specifically, we use a lightweight convolutional neural network as a feature extraction block. To further enhance the representation ability of features, we embed the feature map into two heterogeneous and complementary Riemannian manifold geometric structures, i.e. Grassmannian manifold and symmetric positive definite (SPD) manifold. Then, to facilitate the measurement, we introduce the Riemann kernel function to embed two heterogeneous manifold structures into the high-dimensional Hilbert space for fusion. Finally, we design a learnable distance metric scheme that can be optimized according to the divergence of inter-class pairs and intra-class pairs, thereby reducing the impact of large intra-class differences and high inter-class similarity on scene recognition. We verify the effectiveness of MMML on three commonly used datasets. The results show that the accuracy of our proposed method is from 1.21% to 3.12% higher than the state-of-the-art methods.