Find a Way Forward: a Language-Guided Semantic Map Navigator

Understand

In this paper, we introduce the map-language navigation task where an agent executes natural language instructions and moves to the target position based only on a given 3D semantic map.

• To tackle the task, we design the instruction-aware Path Proposal and Discrimination model (iPPD).
• Our approach leverages map information to provide instruction-aware path proposals, i.e., it selects all potential instruction-aligned candidate paths to reduce the solution space.
• Next, to represent the map observations along a path for a better modality alignment, a novel Path Feature Encoding scheme tailored for semantic maps is proposed.