HAMNet: hyperspectral image classification based on hybrid neural network with attention mechanism and multi-scale feature fusion

Abstract

Recently, convolutional neural network (CNN) has made great progress in hyperspectral image (HSI) classification. Considering the problems of high dimensions, limited training samples and intra-class variations of hyperspectral data, there are challenges for traditional pure 2D or 3D deep convolutional neural networks in classifying HSI. Deeper layers bring gradient dispersion, while 3D feature blocks bring a large number of parameters during feature fusion. In this paper, an end-to-end hybrid convolutional neural network is proposed for HSI classification. Firstly, 3D, 2D and 1D convolution modules are applied, respectively, to perform joint feature extraction of spatial and spectral information. Secondly, a new 3D multi-scale feature fusion strategy is proposed to fuse the high-level and low-level features for ensuring the feature sufficiency. Moreover, channel attention mechanism is introduced to avoid feature channel redundancy and strengthen effective features. Comparative experimental results show that the method can receive satisfactory results on public data sets and small-sample learning problem.