2019

Learning to Speak and Act in a Fantasy Text Adventure Game

Urbanek, Jack, Fan, Angela, Karamcheti, Siddharth et al.

Understand

We introduce a large scale crowdsourced text adventure game as a research platform for studying grounded dialogue.

In it, agents can perceive, emote, and act whilst conducting dialogue with other agents.
Models and humans can both act as characters within the game.
We describe the results of training state-of-the-art generative and retrieval models in this setting.

Built on

Transfertransfo: A transfer learning approach for neural network based conversational agents
Original
Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019 · 1901
Earlier work this paper cites.
The second conversational intelligence challenge (convai2)
Original
Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, et al. 2019a · 1902
Earlier work this paper cites.
Intelligence without representation
Rodney A Brooks. 1991 · 1991
Earlier work this paper cites.
Intentional context in situated natural language learning
Michael Fleischman and Deb Roy. 2005 · 2005
Earlier work this paper cites.
Visualizing data using t-sne
Laurens van der Maaten and Geoffrey E. Hinton. 2008 · 2008
Earlier work this paper cites.
Multi-user virtual environments for teaching and learning
Edward Dieterle. 2009 · 2009
Earlier work this paper cites.
Towards understanding situated natural language
Antoine Bordes, Nicolas Usunier, Ronan Collobert, and Jason Weston. 2010 · 2010
Earlier work this paper cites.
The second dialog state tracking challenge
Matthew Henderson, Blaise Thomson, and Jason D Williams. 2014 · 2014
Earlier work this paper cites.
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014 · 2014
Earlier work this paper cites.
Glove: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014 · 2014
Earlier work this paper cites.
Vqa: Visual question answering
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015 · 2015
Earlier work this paper cites.
Language understanding for text-based games using deep reinforcement learning
Original
Karthik Narasimhan, Tejas Kulkarni, and Regina Barzilay. 2015 · 2015
Earlier work this paper cites.
A neural network approach to context-sensitive generation of conversational responses
Original
Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015 · 2015
Earlier work this paper cites.
A neural conversational model
Oriol Vinyals and Quoc Le. 2015 · 2015
Earlier work this paper cites.
A paradigm for situated and goal-driven language learning
Original
Jon Gauthier and Igor Mordatch. 2016 · 2016
Earlier work this paper cites.

Similar

The malmo platform for artificial intelligence experimentation
Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. 2016 · 2016
Cited alongside, same era.
Bag of tricks for efficient text classification
Original
Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016 · 2016
Cited alongside, same era.
Exploring the limits of language modeling
Original
Rafal Jozefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016 · 2016
Cited alongside, same era.
Virtual embodiment: A scalable long-term strategy for artificial intelligence research
Original
Douwe Kiela, Luana Bulat, Anita L Vero, and Stephen Clark. 2016 · 2016
Cited alongside, same era.
A roadmap towards machine intelligence
Tomas Mikolov, Armand Joulin, and Marco Baroni. 2016 · 2016
Cited alongside, same era.
Neural machine translation of rare words with subword units
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 · 2016
Cited alongside, same era.
Learning language games through interaction
Original
Sida I Wang, Percy Liang, and Christopher D Manning. 2016 · 2016
Cited alongside, same era.
Video paragraph captioning using hierarchical recurrent neural networks
Haonan Yu, Jiang Wang, Zhiheng Huang, Yi Yang, and Wei Xu. 2016 · 2016
Cited alongside, same era.
Learning end-to-end goal-oriented dialog
Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017 · 2017
Cited alongside, same era.
Home: A household multimodal environment
Original
Simon Brodeur, Ethan Perez, Ankesh Anand, Florian Golemo, Luca Celotti, Florian Strub, Jean Rouat, Hugo Larochelle, and Aaron Courville. 2017 · 2017
Cited alongside, same era.
Matterport3d: Learning from rgb-d data in indoor environments
Original
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Nießner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017 · 2017
Cited alongside, same era.
Visual dialog
Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José MF Moura, Devi Parikh, and Dhruv Batra. 2017 · 2017
Cited alongside, same era.

Then

Image-grounded conversations: Multimodal context for natural question and response generation
Original
Nasrin Mostafazadeh, Chris Brockett, Bill Dolan, Michel Galley, Jianfeng Gao, Georgios P Spithourakis, and Lucy Vanderwende. 2017 · 2017
Later among the works it cites.
Zero-shot task generalization with multi-task deep reinforcement learning
Original
Junhyuk Oh, Satinder Singh, Honglak Lee, and Pushmeet Kohli. 2017 · 2017
Later among the works it cites.
Minos: Multimodal indoor simulator for navigation in complex environments
Original
Manolis Savva, Angel X Chang, Alexey Dosovitskiy, Thomas Funkhouser, and Vladlen Koltun. 2017 · 2017
Later among the works it cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Later among the works it cites.
Naturalizing a programming language via interactive learning
Original
Sida I Wang, Samuel Ginn, Percy Liang, and Christoper D Manning. 2017 · 2017
Later among the works it cites.
Mastering the dungeon: Grounded language learning by mechanical turker descent
Original
Zhilin Yang, Saizheng Zhang, Jack Urbanek, Will Feng, Alexander H Miller, Arthur Szlam, Douwe Kiela, and Jason Weston. 2017 · 2017
Later among the works it cites.
Textworld: A learning environment for text-based games
Original
Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, et al. 2018 · 2018
Later among the works it cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Later among the works it cites.
Training Millions of Personalized Dialogue Agents
Original
P.-E. Mazaré, S. Humeau, M. Raison, and A. Bordes. 2018 · 2018
Later among the works it cites.
Game-based video-context dialogue
Original
Ramakanth Pasunuru and Mohit Bansal. 2018 · 2018
Later among the works it cites.
Engaging image chat: Modeling personality in grounded dialogue
Original
Kurt Shuster, Samuel Humeau, Antoine Bordes, and Jason Weston. 2018 · 2018
Later among the works it cites.
Talk the walk: Navigating new york city through grounded dialogue
Original
Harm de Vries, Kurt Shuster, Dhruv Batra, Devi Parikh, Jason Weston, and Douwe Kiela. 2018 · 2018
Later among the works it cites.
Neural-symbolic vqa: Disentangling reasoning from vision and language understanding
Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum. 2018 · 2018
Later among the works it cites.
Personalizing dialogue agents: I have a dog, do you have pets too?
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018 · 2018
Later among the works it cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…