2018

FlowQA: Grasping Flow in History for Conversational Machine Comprehension

Huang, Hsin-Yuan, Choi, Eunsol, Yih, Wen-tau

Understand

Conversational machine comprehension requires the understanding of the conversation history, such as previous question/answer pairs, the document context, and the current question.

To enable traditional, single-turn models to encode the history comprehensively, we introduce Flow, a mechanism that can incorporate intermediate representations generated during the process of answering previous questions, through an alternating parallel processing structure.
Compared to approaches that concatenate previous questions/answers as input, Flow integrates the latent semantics of the conversation history more deeply.
Our model, FlowQA, shows superior performance on two recently proposed conversational challenges (+7.2% F1 on CoQA and +4.0% on QuAC).

Built on

Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Data-driven response generation in social media
Alan Ritter, Colin Cherry, and William B. Dolan · 2011
Earlier work this paper cites.
GloVe: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher Manning · 2014
Earlier work this paper cites.
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
End-to-end memory networks with knowledge carryover for multi-turn spoken language understanding
Yun-Nung Chen, Dilek Z. Hakkani-Tür, Gökhan Tür, Jianfeng Gao, and Li Deng · 2016
Earlier work this paper cites.
A theoretically grounded application of dropout in recurrent neural networks
Yarin Gal and Zoubin Ghahramani · 2016
Earlier work this paper cites.
Simpler context-dependent logical forms via model projections
Reginald Long, Panupong Pasupat, and Percy Liang · 2016
Earlier work this paper cites.
Bidirectional attention flow for machine comprehension
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi · 2016
Earlier work this paper cites.
Learning end-to-end goal-oriented dialog
Antoine Bordes, Y-Lan Boureau, and Jason Weston · 2017
Earlier work this paper cites.
Reading Wikipedia to answer open-domain questions
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes · 2017
Earlier work this paper cites.

Similar

From language to programs: Bridging reinforcement learning and maximum marginal likelihood
Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, and Percy Liang · 2017
Cited alongside, same era.
Search-based neural structured learning for sequential question answering
Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang · 2017
Cited alongside, same era.
Deal or no Deal? End-to-end learning for negotiation dialogues
Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra · 2017
Cited alongside, same era.
Adversarial learning for neural dialogue generation
Jiwei Li, Will Monroe, Tianlin Shi, Alan Ritter, and Daniel Jurafsky · 2017
Cited alongside, same era.
Learned in translation: Contextualized word vectors
Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher · 2017
Cited alongside, same era.
Machine comprehension using match-LSTM and answer pointer
Shuohang Wang and Jing Jiang · 2017
Cited alongside, same era.
Gated self-matching networks for reading comprehension and question answering
Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou · 2017
Cited alongside, same era.
QuAC : Question answering in context
Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer · 2018
Cited alongside, same era.
A dataset and baselines for sequential open-domain question answering
Ahmed Elgohary, Chen Zhao, and Jordan Boyd-Garber · 2018
Cited alongside, same era.
Unified pragmatic models for generating and following instructions
Daniel Fried, Jacob Andreas, and Dan Klein · 2018
Cited alongside, same era.
A knowledge-grounded neural conversation model
Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, William B. Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley · 2018
Cited alongside, same era.

Then

FusionNet: Fusing via fully-aware attention with application to machine comprehension
Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, and Weizhu Chen · 2018
Closest in time.
A hierarchical latent structure for variational conversation modeling
Yookoon Park, Jaemin Cho, and Gunhee Kim · 2018
Closest in time.
Deep contextualized word representations
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer · 2018
Closest in time.
Interpretation of natural language rules in conversational machine reading
Marzieh Saeidi, Max Bartolo, Patrick Lewis, Sameer Signh, Tim Rocktäschel, Mike Sheldon, Guillaume Bouchard, and Sebastian Riedel · 2018
Closest in time.
Complex sequential question answering: Towards learning to converse over linked question answer pairs with a knowledge graph
Amrita Saha, Vardaan Pahuja, Mitesh M. Khapra, Karthik Sankaranarayanan, and Sarath Chandar · 2018
Closest in time.
Situated mapping of sequential instructions to actions with single-step reward observation
Alane Suhr and Yoav Artzi · 2018
Closest in time.
Learning to map context-dependent sentences to executable formal queries
Alane Suhr, Srinivasan Iyer, and Yoav Artzi · 2018
Closest in time.
The Web as knowledge-base for answering complex questions
A. Talmor and J. Berant · 2018
Closest in time.
A qualitative comparison of CoQA, SQuAD 2.0 and QuAC
Original
Mark Yatskar · 2018
Closest in time.
QANet: Combining local convolution with global self-attention for reading comprehension
Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le · 2018
Closest in time.
CoQA: A conversational question answering challenge
Siva Reddy, Danqi Chen, and Christopher D. Manning · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…