Topological Planning with Transformers for Vision-and-Language Navigation

Understand

Conventional approaches to vision-and-language navigation (VLN) are trained end-to-end but struggle to perform well in freely traversable environments.

• Inspired by the robotics community, we propose a modular approach to VLN using topological maps.
• Given a natural language instruction and topological map, our approach leverages attention mechanisms to predict a navigation plan in the map.
• The plan is then executed with low-level actions (e.g.