Fetching the paper…
Reading the bibliography…
Retrieval-augmented language models can better adapt to changes in world state and incorporate long-tail knowledge.
Language models as knowledge bases?
Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel · 1909
Earlier work this paper cites.
How can we know what language models know?
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig · 1911
Earlier work this paper cites.
oLMpics– on what language model pre-training captures
Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant · 1912
Earlier work this paper cites.
A Statistical Interpretation of Term Specificity and its Application in Retrieval
Karen Spärck Jones · 1972
Earlier work this paper cites.
Estimating the Dimension of a Model
Gideon Schwarz · 1978
Earlier work this paper cites.
Okapi at TREC-3
Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al · 1995
Earlier work this paper cites.
On the Surprising Behavior of Distance Metrics in High Dimensional Space
Charu C Aggarwal, Alexander Hinneburg, and Daniel A Keim · 2001
Earlier work this paper cites.
Retrieval Augmented Language Model Pre-Training
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang · 2002
Earlier work this paper cites.
Longformer: The Long-document Transformer, 2020
Iz Beltagy, Matthew E. Peters, and Arman Cohan · 2004
Earlier work this paper cites.
ColBERT: Efficient and effective passage search via contextualized late interaction over bert
Omar Khattab and Matei Zaharia · 2004
Earlier work this paper cites.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al · 2005
Earlier work this paper cites.
The Probabilistic Relevance Framework: BM25 and Beyond
Stephen Robertson, Hugo Zaragoza, et al · 2009
Earlier work this paper cites.
Distilling Knowledge from Reader to Retriever for Question Answering, 2022
Gautier Izacard and Edouard Grave · 2012
Earlier work this paper cites.
Reading Wikipedia to Answer Open-Domain Questions
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes · 2017
Earlier work this paper cites.
Contextualizing citations for scientific summarization using word embeddings and domain knowledge
Arman Cohan and Nazli Goharian · 2017
Earlier work this paper cites.
Stefanos Angelidis and Mirella Lapata · 2018
Earlier work this paper cites.
The NarrativeQA Reading Comprehension Challenge
Tomáš Kočiskỳ, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette · 2018
Earlier work this paper cites.
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction, 2018
Leland McInnes, John Healy, and James Melville · 2018
Earlier work this paper cites.
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension, 2018
Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le · 2018
Earlier work this paper cites.
Transformer-XL: Attentive language models beyond a fixed-length context
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc Le, and Ruslan Salakhutdinov · 2019
Earlier work this paper cites.
Billion-Scale Similarity Search with GPUs
Jeff Johnson, Matthijs Douze, and Hervé Jégou · 2019
Earlier work this paper cites.
Sentence-BERT: Sentence embeddings using Siamese BERT-networks
Nils Reimers and Iryna Gurevych · 2019
Cited alongside, same era.
Language Models are Few-Shot Learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Cited alongside, same era.
Dense Passage Retrieval for Open-Domain Question Answering
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih · 2020
Cited alongside, same era.
UNIFIEDQA: Crossing format boundaries with a single QA system
Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi · 2020
Cited alongside, same era.
Frustratingly hard evidence retrieval for QA over books
Xiangyang Mou, Mo Yu, Bingsheng Yao, Chenghao Yang, Xiaoxiao Guo, Saloni Potdar, and Hui Su · 2020
Cited alongside, same era.
LlamaIndex, 2022
Jerry Liu · 2022
Later among the works it cites.
Memory-based model editing at scale
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn · 2022
Later among the works it cites.
QuALITY: Question Answering with Long Input Texts, Yes!
Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, and Samuel Bowman · 2022
Later among the works it cites.
Recitation-augmented language models
Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou · 2022
Later among the works it cites.
Generate rather than retrieve: Large Language Models are strong context generators, 2022
Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
How Much Knowledge Can You Pack Into the Parameters of a Language Model?
Adam Roberts, Colin Raffel, and Noam Shazeer · 2020
Cited alongside, same era.
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner · 2021
Cited alongside, same era.
Dense hierarchical retrieval for open-domain question answering
Ye Liu, Kazuma Hashimoto, Yingbo Zhou, Semih Yavuz, Caiming Xiong, and Philip Yu · 2021
Cited alongside, same era.
Joint passage ranking for diverse multi-answer retrieval
Sewon Min, Kenton Lee, Ming-Wei Chang, Kristina Toutanova, and Hannaneh Hajishirzi · 2021
Cited alongside, same era.
Scaling language models: Methods, Analysis & Insights from Training Gopher
Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al · 2021
Cited alongside, same era.
Do long-range language models actually use long-range context?
Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, and Mohit Iyyer · 2021
Cited alongside, same era.
Recursively Summarizing Books with Human Feedback, 2021
Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, and Paul Christiano · 2021
Cited alongside, same era.
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, et al · 2023
Later among the works it cites.
Hybrid hierarchical retrieval for open-domain question answering
Manoj Ghuhan Arivazhagan, Lan Liu, Peng Qi, Xinchi Chen, William Yang Wang, and Zhiheng Huang · 2023
Later among the works it cites.
Sparks of Artificial General Intelligence: Early Experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al · 2023
Later among the works it cites.
CoLISA: Inner Interaction via Contrastive Learning for Multi-choice Reading Comprehension
Mengxing Dong, Bowei Zou, Yanling Li, and Yu Hong · 2023
Later among the works it cites.
Enabling large language models to generate text with citations
Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen · 2023
Later among the works it cites.
Large Language Models struggle to learn Long-Tail Knowledge
Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel · 2023
Later among the works it cites.
Lost in the middle: How language models use long contexts
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang · 2023
Later among the works it cites.
Nonparametric masked language modeling
Sewon Min, Weijia Shi, Mike Lewis, Xilun Chen, Wen-tau Yih, Hannaneh Hajishirzi, and Luke Zettlemoyer · 2023
Later among the works it cites.
A neural CRF-based hierarchical approach for linear text segmentation
Inderjeet Nair, Aparna Garimella, Balaji Vasan Srinivasan, Natwar Modani, Niyati Chhaya, Srikrishna Karanam, and Sumit Shekhar · 2023
Later among the works it cites.
A controllable qa-based framework for decontextualization
Benjamin Newman, Luca Soldaini, Raymond Fok, Arman Cohan, and Kyle Lo · 2023
Later among the works it cites.
OpenAI · 2023
Later among the works it cites.
In-context retrieval-augmented language models
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham · 2023
Later among the works it cites.
Questions are all you need to train a dense passage retriever
Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau, and Manzil Zaheer · 2023
Later among the works it cites.
Shall we pretrain autoregressive language models with retrieval? a comprehensive study
Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, et al · 2023
Later among the works it cites.
Extractive is not faithful: An investigation of broad unfaithfulness problems in extractive summarization
Shiyue Zhang, David Wan, and Mohit Bansal · 2023
Later among the works it cites.