Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution

Abstract

This paper focuses on hyperspectral image (HSI) super-resolution that aims to\nfuse a low-spatial-resolution HSI and a high-spatial-resolution multispectral\nimage to form a high-spatial-resolution HSI (HR-HSI). Existing deep\nlearning-based approaches are mostly supervised that rely on a large number of\nlabeled training samples, which is unrealistic. The commonly used model-based\napproaches are unsupervised and flexible but rely on hand-craft priors.\nInspired by the specific properties of model, we make the first attempt to\ndesign a model inspired deep network for HSI super-resolution in an\nunsupervised manner. This approach consists of an implicit autoencoder network\nbuilt on the target HR-HSI that treats each pixel as an individual sample. The\nnonnegative matrix factorization (NMF) of the target HR-HSI is integrated into\nthe autoencoder network, where the two NMF parts, spectral and spatial\nmatrices, are treated as decoder parameters and hidden outputs respectively. In\nthe encoding stage, we present a pixel-wise fusion model to estimate hidden\noutputs directly, and then reformulate and unfold the model's algorithm to form\nthe encoder network. With the specific architecture, the proposed network is\nsimilar to a manifold prior-based model, and can be trained patch by patch\nrather than the entire image. Moreover, we propose an additional unsupervised\nnetwork to estimate the point spread function and spectral response function.\nExperimental results conducted on both synthetic and real datasets demonstrate\nthe effectiveness of the proposed approach.\n