Local Auxiliary Spatial–Spectral Decoupling Transformer Network for Cross-Scene Hyperspectral Image Classification

Abstract

The feature-level domain alignment based on deep learning techniques has greatly improved the performance of unsupervised domain adaptation (UDA) for hyperspectral image (HSI) classification. However, most of these methods leverage convolutional neural networks to capture local features, overlooking the comparable spatial global (SaG) and spectral global (SeG) information shared by both the source and target domains. To overcome this issue, we propose a local auxiliary spatial–spectral decoupling transformer network to ease the learning of global domain-invariant information. The SaG and SeG features of HSIs are extracted through a dual-branch design, preventing the feature coupling of different dimensions. In order to compress the model’s parameter search space, a local auxiliary global feature extraction strategy is devised. First, local prior constraints are introduced by extracting primitive features using a convolutional intra-token embedding. Next, the extraction of global spatial and spectral information from these primitive features is effectively achieved using the self-attention mechanism. Finally, a dynamic feature fusion mechanism is devised that enables the model to focus on features more conducive to transfer while suppressing irrelevant features. By using only standard adversarial domain alignment, LASDT achieves the state-of-the-art performance, demonstrating the model’s superior capability in UDA for HSI classification.