Deep Multiscale Fine-Grained Hashing for Remote Sensing Cross-Modal Retrieval

Abstract

Hashing retrieval is a widely used technique in high spatial resolution remote sensing images due to its efficient retrieval speed and low memory overhead. However, existing hashing retrieval methods primarily focus on matching multi-label remote sensing images, neglecting the extensive fine-grained semantic information in cross-modal remote sensing data. Moreover, remote sensing images exhibit noticeable object size differences and contain redundant features that lack effective multiscale feature extraction methods. To address these issues, we propose a novel Deep Multiscale Fine-grained Hashing (DMFH) method for cross-modal hashing retrieval of remote sensing data. The DMFH method comprises two modules: the feature extraction module and hashing retrieval module. In the feature extraction module, we introduce a multiscale feature representation method to extract both low-level and high-level features from remote sensing images while employing a redundant optimizer to remove duplicate features. Additionally, we utilized embedding vectors to extract fine-grained semantic information from description texts. The hashing retrieval module employs contrastive loss and triplet loss to guide the hash function toward learning and generating hash codes from extracted features. Our proposed DMFH method achieves state-of-the-art performance in two public remote sensing image-text datasets(RSICD and RSITMD) through extensive experiments and ablation studies.