Triple-branch ternary-attention mechanism network with deformable 3D convolution for hyperspectral image classification

Abstract

In recent years, the classification of hyperspectral images (HSI) has received extensive research attention. As compared with traditional HSI classification, which only uses spectral information, it is found that spatial information is also essential in HSI classification. To effectively utilize the spectral and spatial information of HSI, this paper proposes a triple-branch ternary-attention mechanism network with deformable 3D convolution (D3DTBTA). In D3DTBTA, three branches, i.e. the spectral, spatial-X, and spatial-Y branches, are combined with the attention mechanism in three directions, which can better capture the vector features of three dimensions in HSI. Furthermore, considering the adaptation of scale and receptive field size in the convolution operation, our method uses deformable convolution to enable D3DTBTA to enhance feature extraction. Our experimental results show that the framework outperforms existing algorithms on four hyperspectral datasets, especially when the training samples are limited.