Hyperspectral image super-resolution employing nonlocal block and hybrid multiscale three-dimensional convolution

Abstract

Due to the special characteristics of hyperspectral images (HSIs), hundreds of continuous bands, and low spatial resolution, it is of great importance to explore the coherence among hyperspectral bands and extract the spatial information as far as possible to reconstruct the high-resolution (HR) HSIs, in which most of the methods failed. We propose an HSI super-resolution (SR) method termed NLB-HMS3D, which consists of two main parts named the spatial–similarity features module and the spatial and spectral correlation utilization module. Different from the majority of existing methods that stack multiple parallel two-dimensional convolution layers to blindly extract more spatial features, we introduce the nonlocal block to expand the receptive fields to thoroughly dig the spatial–similarity features from the image itself. This block not only greatly improves the effectiveness but also reduces tons of parameters. To better preserve the spectral details, we further propose a new block called multiscale spectral features fusion block using the separated three-dimensional convolution with different convolution kernel sizes to explore the diverse spatial–spectral features and fuse them to recover better spectral details. The experiments and data analysis demonstrate that NLB-MS3D can obtain superior performance over many existing state-of-the-art algorithms.