Learning Feature Descriptors using Camera Pose Supervision

Understand

Recent research on learned visual descriptors has shown promising improvements in correspondence estimation, a key component of many 3D vision tasks.

• However, existing descriptor learning frameworks typically require ground-truth correspondences between feature points for training, which are challenging to acquire at scale.
• In this paper we propose a novel weakly-supervised framework that can learn feature descriptors solely from relative camera poses between images.
• To do so, we devise both a new loss function that exploits the epipolar constraint given by camera poses, and a new model architecture that makes the whole pipeline differentiable and efficient.