Representation Learning for Grounded Spatial Reasoning

Understand

The interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment.

• We consider the task of spatial reasoning in a simulated environment, where an agent can act and receive rewards.
• The proposed model learns a representation of the world steered by instruction text.
• This design allows for precise alignment of local neighborhoods with corresponding verbalizations, while also handling global references in the instructions.