PiCovS: Pixel-Level With Covariance Pooling Feature and Superpixel-Level Feature Fusion for Hyperspectral Image Classification

Abstract

In hyperspectral image (HSI) classification, Convolutional Neural Networks (CNNs) have exhibited exceptional performance, owing to their hierarchical nonlinear modeling. However, their fixed square receptive field constrains their ability to effectively handle irregular image regions. Graph Convolution Networks (GCNs) have been introduced to learn irregular regions through correlations between adjacent pixels modeled as superpixel-based nodes, yet they lack pixel-level information. We propose a novel approach "Pixel-level with Covariance Pooling feature and Superpixel-level feature Fusion for Hyperspectral Image Classification" (PiCovS). Our method harnesses complementary spectral-spatial features at both pixel and superpixel levels to capture characteristics of both small-scale regular and large-scale irregular regions. We introduce a hybrid network that integrates and propagates features between image-level pixels and graph-level nodes using a graph encoder-decoder, effectively reconciling the differences between regular CNN and irregular GCN data representations. To enhance superpixel boundary learning, we modify the Manifold Simple Linear Iterative Clustering (M-SLIC) algorithm by incorporating texture feature information, resulting in refined superpixel representations. Additionally, we propose a novel covariance pooling mechanism with an attention mechanism within the CNN branch, enabling the capturing and utilization of holistic HSI information along spectral and spatial dimensions by exploiting second-order statistics throughout the network. Our comprehensive experiments showcase the efficiency and robustness of the proposed framework, achieving an impressive overall accuracy of 99.84%, 99.97%, 99.98%, and 81.96% on the Indian Pines, University of Pavia, Salinas, and the Houston University datasets, respectively. Remarkably, PiCovS excels even with limited training samples, outperforming other state-of-the-art methods in accuracy.