Fetching the paper…

Reading the bibliography…

A Comparative Study of Word Embeddings for Reading Comprehension · Around

2017

A Comparative Study of Word Embeddings for Reading Comprehension

Dhingra, Bhuwan, Liu, Hanxiao, Salakhutdinov, Ruslan et al.

Original
PDF
Post on X

Understand

The focus of past machine learning research for Reading Comprehension tasks has been primarily on the design of novel deep learning architectures.

Here we show that seemingly minor choices made on (1) the use of pre-trained word embeddings, and (2) the representation of out-of-vocabulary tokens at test time, can turn out to have a larger impact than architectural choices on the final performance.
We systematically explore several options for these choices, and provide recommendations to researchers working in this area.

Built on

Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013 · 2013
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Original
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014 · 2014
Earlier work this paper cites.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Original
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014 · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Diederik Kingma and Jimmy Ba. 2014 · 2014
Earlier work this paper cites.
Glove: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014 · 2014
Earlier work this paper cites.
Teaching machines to read and comprehend
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015 · 2015
Earlier work this paper cites.
The goldilocks principle: Reading children’s books with explicit memory representations
Original
Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2015 · 2015
Earlier work this paper cites.
Improving distributional similarity with lessons learned from word embeddings
Omer Levy, Yoav Goldberg, and Ido Dagan. 2015 · 2015
Earlier work this paper cites.
Embracing data abundance: Booktest dataset for reading comprehension
Original
Ondrej Bajgar, Rudolf Kadlec, and Jan Kleindienst. 2016 · 2016
Earlier work this paper cites.

Similar

A thorough examination of the cnn/daily mail reading comprehension task
Original
Danqi Chen, Jason Bolton, and Christopher D Manning. 2016 · 2016
Cited alongside, same era.
Attention-over-attention neural networks for reading comprehension
Original
Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. 2016 · 2016
Cited alongside, same era.
Gated-attention readers for text comprehension
Original
Bhuwan Dhingra, Hanxiao Liu, William W Cohen, and Ruslan Salakhutdinov. 2016 · 2016
Cited alongside, same era.
Tracking the world state with recurrent entity networks
Original
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. 2016 · 2016
Cited alongside, same era.
Text understanding with the attention sum reader network

Then

Squad: 100,000+ questions for machine comprehension of text
Original
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016
Later among the works it cites.
Bidirectional attention flow for machine comprehension
Original
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016 · 2016
Later among the works it cites.
Reasonet: Learning to stop reading in machine comprehension
Original
Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. 2016 · 2016
Later among the works it cites.
Iterative alternating neural attention for machine reading
Original
Alessandro Sordoni, Phillip Bachman, and Yoshua Bengio. 2016 · 2016
Later among the works it cites.
Newsqa: A machine comprehension dataset

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, and Jan Kleindienst. 2016 · 2016

Cited alongside, same era.

Dynamic entity representations with max-pooling improves machine reading

Sosuke Kobayashi, Ran Tian, Naoaki Okazaki, and Kentaro Inui. 2016 · 2016

Cited alongside, same era.

Neural semantic encoders

Tsendsuren Munkhdalai and Hong Yu. 2016 · 2016

Cited alongside, same era.

Ms marco: A human generated machine reading comprehension dataset

Original

Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016 · 2016

Cited alongside, same era.

Who did what: A large-scale person-centered cloze dataset

Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2016 · 2016

Cited alongside, same era.

Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2016 · 2016

Later among the works it cites.

Machine comprehension using match-lstm and answer pointer

Shuohang Wang and Jing Jiang. 2016 · 2016

Later among the works it cites.

Multi-perspective context matching for machine comprehension

Zhiguo Wang, Haitao Mi, Wael Hamza, and Radu Florian. 2016 · 2016

Later among the works it cites.

Dynamic coattention networks for question answering

Caiming Xiong, Victor Zhong, and Richard Socher. 2016 · 2016

Later among the works it cites.

End-to-end answer chunk extraction and ranking for reading comprehension

Yang Yu, Wei Zhang, Kazi Hasan, Mo Yu, Bing Xiang, and Bowen Zhou. 2016 · 2016

Later among the works it cites.