Co-Attention for Conditioned Image Matching

Understand

We propose a new approach to determine correspondences between image pairs in the wild under large changes in illumination, viewpoint, context, and material.

• While other approaches find correspondences between pairs of images by treating the images independently, we instead condition on both images to implicitly take account of the differences between them.
• To achieve this, we introduce (i) a spatial attention mechanism (a co-attention module, CoAM) for conditioning the learned features on both images, and (ii) a distinctiveness score used to choose the best matches at test time.
• CoAM can be added to standard architectures and trained using self-supervision or supervised data, and achieves a significant performance improvement under hard conditions, e.g.