FM-Loc: Using Foundation Models for Improved Vision-based Localization

Understand

Visual place recognition is essential for vision-based robot localization and SLAM.

• Despite the tremendous progress made in recent years, place recognition in changing environments remains challenging.
• A promising approach to cope with appearance variations is to leverage high-level semantic features like objects or place categories.
• In this paper, we propose FM-Loc which is a novel image-based localization approach based on Foundation Models that uses the Large Language Model GPT-3 in combination with the Visual-Language Model CLIP to construct a semantic image descriptor that is robust to severe changes in scene geometry and camera viewpoint.