CycleGAN-STF: Spatiotemporal Fusion via CycleGAN-Based Image Generation

Abstract

Due to the trade-off of temporal resolution and spatial resolution, spatiotemporal image-fusion uses existing high-spatial-low-temporal (HSLT) and high-temporal-low-spatial (HTLS) images as prior knowledge to reconstruct high-temporal-high-spatial (HTHS) images. However, some existing spatiotemporal image-fusion algorithms ignore the issue that the spatial information of HTLS images is insufficient to support the acquisition of spatial information, which leads to the unsatisfactory accuracy of the fusion result. To introduce more spatial information, the algorithm in this article uses Cycle-generative adversarial networks (GANs) to simulate the change process of two HSLT images at k-1 and k+1, and to generate some simulated images between k-1 and k+1. Then, the generated images are selected under the help of HTLS images, and the selected ones are then enhanced with wavelet transform. Finally, the image with spatial information is introduced into the Flexible Spatiotemporal DAta Fusion (FSDAF) framework to improve the performance of spatiotemporal image-fusion. Extensive experiments on two real data sets demonstrate that our proposed method outperforms current state-of-the-art spatiotemporal image-fusion methods.