MatchFormer: Interleaving Attention in Transformers for Feature Matching

Understand

Local feature matching is a computationally intensive task at the subpixel level.

• While detector-based methods coupled with feature descriptors struggle in low-texture scenes, CNN-based methods with a sequential extract-to-match pipeline, fail to make use of the matching capacity of the encoder and tend to overburden the decoder for matching.
• In contrast, we propose a novel hierarchical extract-and-match transformer, termed as MatchFormer.
• Inside each stage of the hierarchical encoder, we interleave self-attention for feature extraction and cross-attention for feature matching, yielding a human-intuitive extract-and-match scheme.