Learning to Learn Words from Visual Scenes

Understand

Language acquisition is the process of learning words from the surrounding scene.

• We introduce a meta-learning framework that learns how to learn word representations from unconstrained scenes.
• We leverage the natural compositional structure of language to create training episodes that cause a meta-learner to learn strong policies for language acquisition.
• Experiments on two datasets show that our approach is able to more rapidly acquire novel words as well as more robustly generalize to unseen compositions, significantly outperforming established baselines.