Asymmetric Bidirectional Fusion Network for Remote Sensing Pansharpening

Abstract

Pansharpening aims to generate a high-resolution multi-spectral (HR-MS) image given a paired panchromatic (PAN) image and low-resolution multi-spectral (LR-MS) image. Though existing pansharpening methods have made remarkable progress, the fusion pipeline does not fully adapt to the distinct characteristics of the PAN and LR-MS images. In this paper, to fully exploit the complementary modality of the two images, we propose a novel and efficient asymmetric bidirectional fusion network (ABFNet). The ABFNet consists of the two customized fusion modules with asymmetric architectures, which aim to reinforce the PAN and LR-MS images respectively. Specifically, the spectral colorization module recalibrates the scale and bias of the PAN features using weights generated by the LR-MS features, which aims to inject spectral information into the PAN features without breaking their spatial continuity. To transfer spatial details from the PAN features into the LR-MS features, the spatial restoration codebook module refines the LR-MS features with point-to-point restoration codebooks learned from the PAN features. By incorporating the two modules in multiple stages, ABFNet enjoys a high capability for capturing both spectral and spatial dependencies. Extensive experiments over multiple satellite datasets demonstrate the effectiveness of the proposed methods.