RDFM: Robust Deep Feature Matching for Multimodal Remote-Sensing Images

Abstract

Robust feature matching for multimodal remote sensing images remains challenging due to the significant nonlinear radiation difference (NRD) caused by modality variations. In this letter, we present a novel feature-matching method for multimodal remote sensing images, called RDFM, which exploits only deep features extracted by a pre-trained VGG network to achieve competitive performance. It is shown that template matching of these pre-trained features is robust to NRD for various multi-modal remote sensing images, and no additional training is required to improve the matching performance. In order to extract as many correspondences as possible, we use dense template matching to obtain point correspondences and introduce a 4D convolution-based implementation of dense template matching for the sake of computational efficiency. RDFM consists of two main steps. First, enormous coarse correspondences are extracted by applying dense template matching at the deep layer of the pre-trained network, and then a coarse-to-fine hierarchical refinement is performed to obtain high-quality correspondences. To verify the effectiveness of RDFM, six different types of multimodal image datasets are used in our experiments, including day-night, depth-optical, infrared-optical, map-optical, optical-optical, and SAR-optical datasets. The comprehensive experimental results show that RDFM is able to overcome the problem caused by NRD and achieves a better performance than the state-of-the-art methods for multimodal remote sensing image matching. The code of RDFM is publicly available at https://github.com/Fans2017/RDFM.