Deep Semantic Feature Matching Using Confidential Correspondence Consistency

Abstract

This work aims to establish visual correspondences between a pair of images depicting objects of the same semantic category. It encounters many challenges such as non-overlapping of scenes or objects, background clutter, and large intra-class variation. Existing methods handle this task with handcrafted features, which cannot effectively fit the correlations between non-overlapping images. Besides, additional training or information may be implemented into the learned features. In this paper, we propose a novel approach for semantic correspondence, which is based on deep feature representation, geometric and semantic associations between intra-class objects, and hierarchical matching selection according to the convolutional feature pyramid. Firstly, we construct the initial correspondence by developing a sparse feature matching model on the coarsest feature level, which enforces the nearest-neighbor searching under semantic and geometric consistency constraints. Further, a narrowing strategy is proposed and employed from the coarsest to the finest feature level, which hierarchically refine and optimize the correspondence. The results illustrate that this approach achieves competitive performance on the public datasets for semantic correspondence.