A Dual-Attention Transformer Network for Pansharpening

Abstract

As a key part of urban sensing, urban remote sensing plays an important role in helping urban planners understand urban issues from a macro-perspective. In urban remote sensing, the pansharpening technique is employed to synthesize high-spatial-resolution (HR) multispectral (MS) images by fusing MS images and panchromatic (PAN) images. However, the existing methods fail to make efficient use of global information. Motivated by the accomplishment of vision Transformer (ViT) in image restoration, a dual-attention Transformer (DAT) module is proposed, and a multiscale U-shaped network, named DAT network (DATN), is set up based on the DAT module. Moreover, skip connection is applied in each level to better transfer information. Different from the previous convolutional neural network (CNN)-based and simply modified Transformer-based methods, DATN is capable of building up long-distance dependencies from local and global perspectives in a computationally friendly pattern. In the end, the proposed DATN is extensively evaluated on three urban remote sensing datasets, including QuickBird (QB), WorldView-2 (WV2), and WorldView-4 (WV4), and the results establish that it surpasses other methods in both objective and visual evaluations.