BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA

A BERT baseline for the natural questions

Original

Chris Alberti, Kenton Lee, and Michael Collins. 2019 · 1901

Earlier work this paper cites.

Roberta: A robustly optimized BERT pretraining approach

Original

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907

Earlier work this paper cites.

Bert is not a knowledge base (yet): Factual knowledge vs. name-based reasoning in unsupervised qa

Original

Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2019 · 1911

Earlier work this paper cites.

REALM: Retrieval-augmented language model pre-training

Original

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020 · 2002

Earlier work this paper cites.

Contextualized representations using textual encyclopedic knowledge

Original

Mandar Joshi, Kenton Lee, Yi Luan, and Kristina Toutanova. 2020 · 2004

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Original

Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020 · 2005

Earlier work this paper cites.

Leveraging passage retrieval with generative models for open domain question answering

Original

Gautier Izacard and Edouard Grave. 2020 · 2007

Earlier work this paper cites.

Aligning books and movies: Towards story-like visual explanations by watching movies and reading books

Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015 · 2015

Earlier work this paper cites.

Improving neural language models with a continuous cache

Original

Edouard Grave, Armand Joulin, and Nicolas Usunier. 2016 · 2016

Earlier work this paper cites.

Commonsense knowledge base completion

Xiang Li, Aynaz Taheri, Lifu Tu, and Kevin Gimpel. 2016 · 2016

Cited alongside, same era.

SQuAD: 100,000+ questions for machine comprehension of text

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Cited alongside, same era.

Joint learning of the embedding of words and entities for named entity disambiguation

Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. 2016 · 2016

Cited alongside, same era.

Reading Wikipedia to answer open-domain questions

Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017 · 2017

Cited alongside, same era.

Billion-scale similarity search with gpus

Original

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017 · 2017

Cited alongside, same era.

Pointer sentinel mixture models

BERT: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019

Later among the works it cites.

Natural questions: A benchmark for question answering research

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019 · 2019

Later among the works it cites.

Knowledge enhanced contextual word representations

Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019 · 2019

Later among the works it cites.

Language models as knowledge bases?

Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019 · 2019

Later among the works it cites.

Language models are unsupervised multitask learners

Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019

Later among the works it cites.