Fetching the paper…

Reading the bibliography…

2020

Generation-Augmented Retrieval for Open-domain Question Answering

Mao, Yuning, He, Pengcheng, Liu, Xiaodong et al.

Original
PDF
Post on X

Understand

We propose Generation-Augmented Retrieval (GAR) for answering open-domain questions, which augments a query through text generation of heuristically discovered relevant contexts without external resources as supervision.

We demonstrate that the generated contexts substantially enrich the semantics of the queries and GAR with sparse representations (BM25) achieves comparable or better performance than state-of-the-art dense retrieval methods such as DPR.
We show that generating diverse contexts for a query is beneficial as fusing their results consistently yields better retrieval accuracy.
Moreover, as sparse and dense representations are often complementary, GAR can be easily combined with DPR to achieve even better performance.

Generation-Augmented Retrieval for Open-domain Question Answering · Around

Built on

Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
Original
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019 · 1910
Earlier work this paper cites.
Learning to retrieve reasoning paths over wikipedia graph for question answering
Original
Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. 2019 · 1911
Earlier work this paper cites.
Knowledge guided text retrieval and reading for open domain question answering
Original
Sewon Min, Danqi Chen, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019b · 1911
Earlier work this paper cites.
Relevance feedback in information retrieval
Joseph Rocchio. 1971 · 1971
Earlier work this paper cites.
Realm: Retrieval-augmented language model pre-training
Original
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020 · 2002
Earlier work this paper cites.
How much knowledge can you pack into the parameters of a language model?
Original
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020 · 2002
Earlier work this paper cites.
Overview of the trec 2019 deep learning track
Original
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M Voorhees. 2020 · 2003
Earlier work this paper cites.
Umass at trec 2004: Novelty and hard
Nasreen Abdul-Jaleel, James Allan, W Bruce Croft, Fernando Diaz, Leah Larkey, Xiaoyan Li, Mark D Smucker, and Courtney Wade. 2004 · 2004
Earlier work this paper cites.
Longformer: The long-document transformer
Original
Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020 · 2004
Earlier work this paper cites.
Dense passage retrieval for open-domain question answering
Original
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2004
Earlier work this paper cites.
Ambigqa: Answering ambiguous open-domain questions
Original
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020 · 2004
Earlier work this paper cites.
Question rewriting for conversational question answering
Original
Svitlana Vakulenko, Shayne Longpre, Zhucheng Tu, and Raviteja Anantha. 2020 · 2004
Earlier work this paper cites.
Language models are few-shot learners
Original
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 2005
Earlier work this paper cites.

Similar

Retrieval-augmented generation for knowledge-intensive nlp tasks
Original
Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020a · 2005
Cited alongside, same era.
Query reformulation using query history for passage retrieval in conversational search
Original
Sheng-Chieh Lin, Jheng-Hong Yang, Rodrigo Nogueira, Ming-Feng Tsai, Chuan-Ju Wang, and Jimmy Lin. 2020 · 2005
Cited alongside, same era.
Sparse, dense, and attentional representations for text retrieval
Original
Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. 2020 · 2005
Cited alongside, same era.
Few-shot generative conversational query rewriting
Original
Shi Yu, Jiahua Liu, Jingqin Yang, Chenyan Xiong, Paul Bennett, Jianfeng Gao, and Zhiyuan Liu. 2020 · 2006
Cited alongside, same era.

Then

Billion-scale similarity search with gpus
Original
Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2017 · 2017
Later among the works it cites.
TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017 · 2017
Later among the works it cites.
Task-oriented query reformulation with reinforcement learning
Rodrigo Nogueira and Kyunghyun Cho. 2017 · 2017
Later among the works it cites.
Anserini: Enabling the use of lucene for information retrieval research
Peilin Yang, Hui Fang, and Jimmy Lin. 2017 · 2017
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Leveraging passage retrieval with generative models for open domain question answering

Original

Gautier Izacard and Edouard Grave. 2020 · 2007

Cited alongside, same era.

Deep reinforced query reformulation for information retrieval

Xiao Wang, Craig Macdonald, and Iadh Ounis. 2020 · 2007

Cited alongside, same era.

Question and answer test-train overlap in open-domain question answering datasets

Patrick Lewis, Pontus Stenetorp, and Sebastian Riedel. 2020b · 2008

Cited alongside, same era.

Reciprocal rank fusion outperforms condorcet and individual rank learning methods

Gordon V Cormack, Charles LA Clarke, and Stefan Buettcher. 2009 · 2009

Cited alongside, same era.

Positional relevance model for pseudo-relevance feedback

Yuanhua Lv and ChengXiang Zhai. 2010 · 2010

Cited alongside, same era.

Constrained abstractive summarization: Preserving factual consistency with constrained generation

Yuning Mao, Xiang Ren, Heng Ji, and Jiawei Han. 2020 · 2010

Cited alongside, same era.

The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems

Original

Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015 · 2015

Cited alongside, same era.

Ms marco: A human-generated machine reading comprehension dataset

Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016 · 2016

Cited alongside, same era.

Natural questions: A benchmark for question answering research

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019 · 2019

Later among the works it cites.

Latent retrieval for weakly supervised open domain question answering

Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019 · 2019

Later among the works it cites.

Generative question refinement with deep reinforcement learning in retrieval-based qa system

Ye Liu, Chenwei Zhang, Xiaohui Yan, Yi Chang, and Philip S Yu. 2019 · 2019

Later among the works it cites.

A discrete hard EM approach for weakly supervised question answering

Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019a · 2019

Later among the works it cites.

Sequence to sequence learning for query expansion

Salah Zaiem and Fatiha Sadat. 2019 · 2019

Later among the works it cites.

The second conversational intelligence challenge (convai2)

Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, et al. 2020 · 2020

Closest in time.

Neurips 2020 efficientqa competition: Systems, analyses and lessons learned

Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, et al. 2021 · 2020

Closest in time.