Learning To Follow Directions in Street View

Understand

Navigating and understanding the real world remains a key challenge in machine learning and inspires a great variety of research in areas such as language grounding, planning, navigation and computer vision.

• We propose an instruction-following task that requires all of the above, and which combines the practicality of simulated environments with the challenges of ambiguous, noisy real world data.
• StreetNav is built on top of Google Street View and provides visually accurate environments representing real places.
• Agents are given driving instructions which they must learn to interpret in order to successfully navigate in this environment.