NCTR: Neighborhood Consensus Transformer for Feature Matching

Abstract

This paper presents NCTR, a feature matcher that enhances input descriptors and finds the correspondences between them. In NCTR, Transformer is applied to aggregate global context for each descriptor. To solve the lack of neighborhood consensus that Transformer may bring, we propose a novel method to evaluate the neighborhood consensus of each key-point and integrate it into the attentional aggregation. The combination of global and local information greatly enhances the model capability and improves the match quality. The experiments on homography estimation and outdoor pose estimation show that NCTR outperforms other hand-designed or learning-based methods and achieves state-of-the-art results.