Neural Refinement for Absolute Pose Regression with Feature Synthesis

Understand

Absolute Pose Regression (APR) methods use deep neural networks to directly regress camera poses from RGB images.

• However, the predominant APR architectures only rely on 2D operations during inference, resulting in limited accuracy of pose estimation due to the lack of 3D geometry constraints or priors.
• In this work, we propose a test-time refinement pipeline that leverages implicit geometric constraints using a robust feature field to enhance the ability of APR methods to use 3D information during inference.
• We also introduce a novel Neural Feature Synthesizer (NeFeS) model, which encodes 3D geometric features during training and directly renders dense novel view features at test time to refine APR methods.