2020

I love your chain mail! Making knights smile in a fantasy game world: Open-domain goal-oriented dialogue agents

Prabhumoye, Shrimai, Li, Margaret, Urbanek, Jack et al.

Understand

Dialogue research tends to distinguish between chit-chat and goal-oriented tasks.

While the former is arguably more naturalistic and has a wider use of language, the latter has clearer metrics and a straightforward learning signal.
Humans effortlessly combine the two, for example engaging in chit-chat with the goal of exchanging information or eliciting a specific response.
Here, we bridge the divide between these two domains in the setting of a rich multi-player text-based fantasy environment where agents and humans engage in both actions and dialogue.

Built on

Reinforcement learning for spoken dialogue systems
Singh, S. P., Kearns, M. J., Litman, D. J., and Walker, M. A · 2000
Earlier work this paper cites.
Optimizing dialogue management with reinforcement learning: Experiments with the njfun system
Singh, S., Litman, D., Kearns, M., and Walker, M · 2002
Earlier work this paper cites.
Let’s go public! taking a spoken dialog system to the real world
Raux, A., Langner, B., Bohus, D., Black, A. W., and Eskenazi, M · 2005
Earlier work this paper cites.
Sample-efficient batch reinforcement learning for dialogue management optimization
Pietquin, O., Geist, M., Chandramohan, S., and Frezza-Buet, H · 2011
Earlier work this paper cites.
Reinforcement learning for adaptive dialogue systems: a data-driven methodology for dialogue management and natural language generation
Rieser, V. and Lemon, O · 2011
Earlier work this paper cites.
Pomdp-based dialogue manager adaptation to extended domains
Gasic, M., Breslin, C., Henderson, M., Kim, D., Szummer, M., Thomson, B., Tsiakoulis, P., and Young, S · 2013
Earlier work this paper cites.
The second dialog state tracking challenge
Henderson, M., Thomson, B., and Williams, J. D · 2014
Earlier work this paper cites.
Deep reinforcement learning in large discrete action spaces
Original
Dulac-Arnold, G., Evans, R., van Hasselt, H., Sunehag, P., Lillicrap, T., Hunt, J., Mann, T., Weber, T., Degris, T., and Coppin, B · 2015
Earlier work this paper cites.
Language understanding for text-based games using deep reinforcement learning
Original
Narasimhan, K., Kulkarni, T., and Barzilay, R · 2015
Earlier work this paper cites.
Policy networks with two-stage training for dialogue systems
Original
Fatemi, M., Asri, L. E., Schulz, H., He, J., and Suleman, K · 2016
Earlier work this paper cites.
Mastering the game of go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al · 2016
Earlier work this paper cites.

Similar

Learning end-to-end goal-oriented dialog
Bordes, A., Boureau, Y.-L., and Weston, J · 2017
Cited alongside, same era.
Learning cooperative visual dialog agents with deep reinforcement learning
Das, A., Kottur, S., Moura, J. M., Lee, S., and Batra, D · 2017
Cited alongside, same era.
Frames: a corpus for adding memory to goal-oriented dialogue systems
El Asri, L., Schulz, H., Sharma, S., Zumer, J., Harris, J., Fine, E., Mehrotra, R., and Suleman, K · 2017
Cited alongside, same era.
Deal or no deal? end-to-end learning for negotiation dialogues
Original
Lewis, M., Yarats, D., Dauphin, Y. N., Parikh, D., and Batra, D · 2017
Cited alongside, same era.
Zero-shot task generalization with multi-task deep reinforcement learning
Oh, J., Singh, S., Lee, H., and Kohli, P · 2017
Cited alongside, same era.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Cited alongside, same era.
Hierarchical text generation and planning for strategic dialogue
Original
Yarats, D. and Lewis, M · 2017
Cited alongside, same era.
Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling
Original
Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., and Gašić, M · 2018
Cited alongside, same era.
Textworld: A learning environment for text-based games
Original
Côté, M.-A., Kádár, Á., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J., Hausknecht, M., Asri, L. E., Adada, M., et al · 2018
Cited alongside, same era.
Wizard of wikipedia: Knowledge-powered conversational agents
Original
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., and Weston, J · 2018
Cited alongside, same era.
Pytorch implementations of reinforcement learning algorithms
Kostrikov, I · 2018
Cited alongside, same era.
A persona-based neural conversation model
Original
Li, J., Galley, M., Brockett, C., Spithourakis, G. P., Gao, J., and Dolan, B
Cited in the paper.

Then

Training millions of personalized dialogue agents
Original
Mazaré, P.-E., Humeau, S., Raison, M., and Bordes, A · 2018
Later among the works it cites.
Learning semantic textual similarity from conversations
Original
Yang, Y., Yuan, S., Cer, D., Kong, S.-y., Constant, N., Pilar, P., Ge, H., Sung, Y.-H., Strope, B., and Kurzweil, R · 2018
Later among the works it cites.
Personalizing dialogue agents: I have a dog, do you have pets too?
Original
Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., and Weston, J · 2018
Later among the works it cites.
Hierarchical decision making by generating and following natural language instructions
Original
Hu, H., Yarats, D., Gong, Q., Tian, Y., and Lewis, M · 2019
Later among the works it cites.
Real-time inference in multi-sentence tasks with deep pretrained transformers
Original
Humeau, S., Shuster, K., Lachaux, M.-A., and Weston, J · 2019
Later among the works it cites.
Countering language drift via visual grounding
Original
Lee, J., Cho, K., and Kiela, D · 2019
Later among the works it cites.
Acute-eval: Improved dialogue evaluation with optimized questions and multi-turn comparisons
Original
Li, M., Weston, J., and Roller, S · 2019
Later among the works it cites.
Deep reinforcement learning for modeling chit-chat dialog with discrete attributes
Original
Sankar, C. and Ravi, S · 2019
Later among the works it cites.
Target-guided open-domain conversation
Tang, J., Zhao, T., Xiong, C., Liang, X., Xing, E., and Hu, Z · 2019
Later among the works it cites.
Learning to speak and act in a fantasy text adventure game
Original
Urbanek, J., Fan, A., Karamcheti, S., Jain, S., Humeau, S., Dinan, E., Rocktäschel, T., Kiela, D., Szlam, A., and Weston, J · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…