2023

Computational Language Acquisition with Theory of Mind

Liu, Andy, Zhu, Hao, Liu, Emmy et al.

Understand

Unlike current state-of-the-art language models, young children actively acquire language through interactions with their surrounding environment and caretakers.

One mechanism that has been argued to be critical to language learning is the ability to infer the mental states of other agents in social environments, coined Theory of Mind (ToM) by Premack & Woodruff (1978).
Drawing inspiration from the modern operationalized versions of ToM implemented in Rabinowitz et al.
(2018) and Zhu et al.

Built on

Sentence-bert: Sentence embeddings using siamese bert-networks
Original
Nils Reimers and Iryna Gurevych · 1908
Earlier work this paper cites.
Does the chimpanzee have a theory of mind?
David Premack and Guy Woodruff · 1978
Earlier work this paper cites.
Bleu: A method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu · 2002
Earlier work this paper cites.
A unified model of early word learning: Integrating statistical and social cues
Chen Yu and Dana H. Ballard · 2004
Earlier work this paper cites.
Multi-agent communication meets natural language: Synergies between functional and structural language learning
Original
Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman · 2005
Earlier work this paper cites.
Constructing a Language: A Usage-Based Theory of Language Acquisition , volume 1
Michael Tomasello · 2005
Earlier work this paper cites.
Emergent multi-agent communication in the deep learning era
Original
Angeliki Lazaridou and Marco Baroni · 2006
Earlier work this paper cites.
Infants rapidly learn word-referent mappings via cross-situational statistics
Linda Smith and Chen Yu · 2007
Earlier work this paper cites.
Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit
Steven Bird, Ewan Klein, and Edward Loper · 2009
Earlier work this paper cites.
Embodied attention and word learning by toddlers
Chen Yu and Linda B. Smith · 2012
Earlier work this paper cites.
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick · 2014
Earlier work this paper cites.

Similar

Reasoning about pragmatics with neural listeners and speakers
Jacob Andreas and Dan Klein · 2016
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Multi-agent cooperation and the emergence of (natural) language
Original
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni · 2016
Cited alongside, same era.
Semantic specificity in one-year-olds’ word comprehension
Elika Bergelson and Richard Aslin · 2017
Cited alongside, same era.
Colors in context: A pragmatic neural model for grounded language understanding
Will Monroe, Robert X.D. Hawkins, Noah D. Goodman, and Christopher Potts · 2017
Cited alongside, same era.
Proximal policy optimization algorithms, 2017
Original
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017
Cited alongside, same era.
Context-aware captions from context-agnostic supervision
Original
Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, and Gal Chechik · 2017
Cited alongside, same era.
Emergent communication through negotiation
Original
Kris Cao, Angeliki Lazaridou, Marc Lanctot, Joel Z Leibo, Karl Tuyls, and Stephen Clark · 2018
Cited alongside, same era.
Sentence-level fluency evaluation: References help, but can be spared!
Katharina Kann, Sascha Rothe, and Katja Filippova · 2018
Cited alongside, same era.
Evaluating theory of mind in question answering
Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Tom Griffiths · 2018
Cited alongside, same era.
Machine theory of mind
Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, SM Ali Eslami, and Matthew Botvinick · 2018
Cited alongside, same era.
Nature and origins of the lexicon in 6-mo-olds
Elika Bergelson and Richard N. Aslin
Cited in the paper.

Then

On the interaction between supervision and self-play in emergent communication
Ryan Lowe, Abhinav Gupta, Jakob Foerster, Douwe Kiela, and Joelle Pineau · 2019
Later among the works it cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Later among the works it cites.
The hanabi challenge: A new frontier for ai research
Nolan Bard, Jakob N Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, et al · 2020
Later among the works it cites.
spacy: Industrial-strength natural language processing in python
Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd · 2020
Later among the works it cites.
Ro{bert}a: A robustly optimized {bert} pretraining approach, 2020
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov · 2020
Later among the works it cites.
Emergent communication of generalizations
Jesse Mu and Noah Goodman · 2021
Later among the works it cites.
Learning transferable visual models from natural language supervision, 2021
Original
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever · 2021
Later among the works it cites.
Few-shot language coordination by modeling theory of mind
Hao Zhu, Graham Neubig, and Yonatan Bisk · 2021
Later among the works it cites.
Emergent communication at scale
Rahma Chaabouni, Florian Strub, Florent Altché, Eugene Tarassov, Corentin Tallec, Elnaz Davoodi, Kory Wallace Mathewson, Olivier Tieleman, Angeliki Lazaridou, and Bilal Piot · 2022
Later among the works it cites.
When and why vision-language models behave like bags-of-words, and what to do about it?, 2022
Original
Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou · 2022
Later among the works it cites.
Simulated language learning through communicative goals and linguistic input
Hao Zhu, Yonatan Bisk, and Graham Neubig · 2022
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…