UNMuTe: Unifying Navigation and Multimodal Dialogue-like Text Generation

Understand

Smart autonomous agents are becoming increasingly important in various real-life applications, including robotics and autonomous vehicles.

• One crucial skill that these agents must possess is the ability to interact with their surrounding entities, such as other agents or humans.
• In this work, we aim at building an intelligent agent that can efficiently navigate in an environment while being able to interact with an oracle (or human) in natural language and ask for directions when it is unsure about its navigation performance.
• The interaction is started by the agent that produces a question, which is then answered by the oracle on the basis of the shortest trajectory to the goal.