Spectral–Spatial Adversarial Multidomain Synthesis Network for Cross-Scene Hyperspectral Image Classification

Abstract

Cross-scene hyperspectral image (HSI) classification has received widespread attention due to its practicality. However, domain adaptation-based cross-scene HSI classification methods are typically tailored for a specific target scene involved in model training and require retraining for new scenes. We instead propose an novel spectral-spatial adversarial multi-domain synthetic network (S ² AMSnet) that can be trained on a single source domain (SD) and generalized to unseen domains. S ² AMSnet improves the robustness of the model to the unseen domain by expanding the diverse distribution of the SD. Specifically, to spatially and spectrally generate diversified generative domain (GD), the spectral-spatial domain generation network (S ² DGN) is designed, and two S ² DGNs with the same structure but not shared parameters are enabled to generate diversified GD through two-step min-max strategy. A Multi-domain mixing module is employed to expand the diversity of the GD further and enhance their class-domain semantic consistency information. Additionally, a multi-scale mutual information regularization network is used to constrain the S ² DGN so that the intrinsic class semantic information of its generated GD does not deviate from the SD. A Semantic consistency discriminator with spectral-spatial feature extraction capability is utilized to capture class-domain semantic consistency information from diverse GD to obtain cross-domain invariant knowledge. Comparative analysis with eight state-of-the-art transfer learning methods on three real HSI datasets, along with an ablation study, validates the effectiveness of the proposed S ² AMSnet in the cross-scene HSI classification task. The codes of this work will be available at https://github.com/daxichen/S2AMSnet.