2019

Countering Language Drift via Visual Grounding

Lee, Jason, Cho, Kyunghyun, Kiela, Douwe

Understand

Emergent multi-agent communication protocols are very different from natural language and not easily interpretable by humans.

We find that agents that were initially pretrained to produce natural language can also experience detrimental language drift: when a non-linguistic reward is used in a goal-based task, e.g.
some scalar success metric, the communication protocol may easily and radically diverge from natural language.
We recast translation as a multi-agent communication game and examine auxiliary training constraints for their effectiveness in mitigating language drift.

Built on

Individual comparisons by ranking methods
Frank Wilcoxon. 1945 · 1945
Earlier work this paper cites.
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014 · 1958
Earlier work this paper cites.
Trainable greedy decoding for neural machine translation
Jiatao Gu, Kyunghyun Cho, and Victor O. K. Li. 2017 · 1978
Earlier work this paper cites.
VENUS: two-phase machine translation system
Kazunori Muraki. 1986 · 1986
Earlier work this paper cites.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J Williams. 1992 · 1992
Earlier work this paper cites.
The synthetic modeling of language origins
Luc Steels. 1997 · 1997
Earlier work this paper cites.
The evolution of language
Martin A Nowak and David C Krakauer. 1999 · 1999
Earlier work this paper cites.
Spontaneous evolution of linguistic structure-an iterated learning model of the emergence of regularity and irregularity
Simon Kirby. 2001 · 2001
Earlier work this paper cites.
Linguistic evolution through language acquisition
Ted Briscoe. 2002 · 2002
Earlier work this paper cites.
Moses: Open source toolkit for statistical machine translation
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007 · 2007
Earlier work this paper cites.
Signals: Evolution, learning, and information
Brian Skyrms. 2010 · 2010
Earlier work this paper cites.
Wit 3 : Web inventory of transcribed and translated talks
Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012 · 2012
Earlier work this paper cites.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Original
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014 · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba. 2014 · 2014
Earlier work this paper cites.
From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014 · 2014
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015 · 2015
Earlier work this paper cites.
Learning language through pictures
Original
Grzegorz Chrupała, Akos Kádár, and Afra Alishahi. 2015 · 2015
Earlier work this paper cites.

Similar

Sequence level training with recurrent neural networks
Original
Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015 · 2015
Cited alongside, same era.
An actor-critic algorithm for sequence prediction
Original
Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2016 · 2016
Cited alongside, same era.
Grounding distributional semantics in the visual world
Marco Baroni. 2016 · 2016
Cited alongside, same era.
Multi30k: Multilingual english-german image descriptions
Desmond Elliott, Stella Frank, Khalil Sima’an, and Lucia Specia. 2016 · 2016
Cited alongside, same era.
Zero-resource translation with multi-lingual neural machine translation
Original
Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T Yarman Vural, and Kyunghyun Cho. 2016 · 2016
Cited alongside, same era.
Learning to communicate with deep multi-agent reinforcement learning
Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson. 2016 · 2016
Cited alongside, same era.
A paradigm for situated and goal-driven language learning
Original
Jon Gauthier and Igor Mordatch. 2016 · 2016
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016 · 2016
Cited alongside, same era.
Multimodal pivots for image caption translation
Original
Julian Hitschler, Shigehiko Schamoni, and Stefan Riezler. 2016 · 2016
Cited alongside, same era.
Multi-agent cooperation and the emergence of (natural) language
Original
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. 2016 · 2016
Cited alongside, same era.
Neural machine translation of rare words with subword units
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 · 2016
Cited alongside, same era.
Multi-source neural translation
Original
Barret Zoph and Kevin Knight. 2016 · 2016
Cited alongside, same era.

Then

Deep Embodiment: Grounding Semantics in Perceptual Modalities
Douwe Kiela. 2017 · 2017
Later among the works it cites.
Learning visually grounded sentence representations
Original
Douwe Kiela, Alexis Conneau, Allan Jabri, and Maximilian Nickel. 2017 · 2017
Later among the works it cites.
Natural language does not emerge ’naturally’ in multi-agent dialog
Satwik Kottur, José M. F. Moura, Stefan Lee, and Dhruv Batra. 2017 · 2017
Later among the works it cites.
Multi-agent cooperation and the emergence of (natural) language
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. 2017 · 2017
Later among the works it cites.
Deal or no deal? end-to-end learning for negotiation dialogues
Original
Mike Lewis, Denis Yarats, Yann N Dauphin, Devi Parikh, and Dhruv Batra. 2017 · 2017
Later among the works it cites.
Adversarial learning for neural dialogue generation
Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017 · 2017
Later among the works it cites.
Emergence of grounded compositional language in multi-agent populations
Original
Igor Mordatch and Pieter Abbeel. 2017 · 2017
Later among the works it cites.
Zero-resource machine translation by multimodal encoder–decoder network with multimedia pivot
Hideki Nakayama and Noriki Nishida. 2017 · 2017
Later among the works it cites.
Task-oriented query reformulation with reinforcement learning
Rodrigo Nogueira and Kyunghyun Cho. 2017 · 2017
Later among the works it cites.
A deep reinforced model for abstractive summarization
Original
Romain Paulus, Caiming Xiong, and Richard Socher. 2017 · 2017
Later among the works it cites.
Towards neural machine translation with partially aligned corpora
Yining Wang, Yang Zhao, Jiajun Zhang, Chengqing Zong, and Zhengshan Xue. 2017 · 2017
Later among the works it cites.
Zero-resource neural machine translation with multi-agent communication game
Yun Chen, Yang Liu, and Victor O. K. Li. 2018 · 2018
Later among the works it cites.
Emergent language in a multi-modal, multi-step referential game
Katrina Evtimova, Andrew Drozdov, Douwe Kiela, and Kyunghyun Cho. 2018 · 2018
Later among the works it cites.
VSE++: improving visual-semantic embeddings with hard negatives
Fartash Faghri, David J. Fleet, Jamie Kiros, and Sanja Fidler. 2018 · 2018
Later among the works it cites.
Lessons learned in multilingual grounded language learning
Original
Ákos Kádár, Desmond Elliott, Marc-Alexandre Côté, Grzegorz Chrupała, and Afra Alishahi. 2018 · 2018
Later among the works it cites.
Emergent translation in multi-agent communication
Jason Lee, Kyunghyun Cho, Jason Weston, and Douwe Kiela. 2018 · 2018
Later among the works it cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…