Few-Shot Hyperspectral Image Classification Based on Cross-Domain Spectral Semantic Relation Transformer

Abstract

In practical hyperspectral image (HSI) classification tasks, we often encounter the problems of few-shot classification and domain misalignment between source domains and target domains. To solve this classification paradigm, a meta-learning method of few-shot learning (FSL) is usually used. However, most existing FSL methods address the problem for domain alignment and neglect the exploration of semantic relationships of objects across domains. In this paper, we propose the cross-domain spectral semantic transformer FSL (SSTFSL), which can fully extract semantic features and spectral detail features for the cross-domain few-shot HSI classification task. Specifically, the multi-head self-attention (MSA) mechanism with enhancement process (EP) of the transformer is used to map out semantically relevant local regions and can enhance the ability of the model to distinguish subtle feature differences in the spectrum. In addition, the matching degree of different branches is computed by relational network learning, which ultimately enables cross-domain few-shot HSI classification. Through extensive experiments, we evaluate the classification performance of SSTFSL on HSI datasets. The results demonstrate that SSTFSL outperforms existing FSL methods and deep learning methods on HSI classification.