Fetching the paper…
Reading the bibliography…
Retrieval-augmented generation improves various aspects of large language models (LLMs) generation, but suffers from computational overhead caused by long contexts as well as the propagation of irrelevant retrieved information into generated responses.
Overview of the trec 2019 deep learning track, 2020
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees · 2003
Earlier work this paper cites.
Improving efficient neural ranking models with cross-architecture knowledge distillation, 2021
Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury · 2010
Earlier work this paper cites.
Ms marco: A human generated machine reading comprehension dataset
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng · 2016
Earlier work this paper cites.
HotpotQA: A dataset for diverse, explainable multi-hop question answering
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning · 2018
Earlier work this paper cites.
Multi-step retriever-reader interaction for scalable open-domain question answering
Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum · 2019
Earlier work this paper cites.
Natural questions: a benchmark for question answering research
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al · 2019
Earlier work this paper cites.
PyTorch: an imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala · 2019
Earlier work this paper cites.
Real-time open-domain question answering with dense-sparse phrase index
Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur Parikh, Ali Farhadi, and Hannaneh Hajishirzi · 2019
Earlier work this paper cites.
TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages
Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki · 2020
Earlier work this paper cites.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela · 2020
Earlier work this paper cites.
Passage re-ranking with bert, 2020
Rodrigo Nogueira and Kyunghyun Cho · 2020
Earlier work this paper cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush · 2020
Earlier work this paper cites.
Decontextualization: Making sentences stand-alone
Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski, Dipanjan Das, and Michael Collins · 2021
Earlier work this paper cites.
Ms marco: Benchmarking ranking models in the large-data regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Jimmy Lin · 2021
Earlier work this paper cites.
Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations
Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira · 2021
Earlier work this paper cites.
BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models
Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych · 2021
Earlier work this paper cites.
Atlas: Few-shot Learning with Retrieval Augmented Language Models, November 2022
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave · 2022
Earlier work this paper cites.
Retromae: Pre-training retrieval-oriented language models via masked auto-encoder
Xiao Shitao, Liu Zheng, Shao Yingxia, and Cao Zhao · 2022
Cited alongside, same era.
Dynamic context pruning for efficient and interpretable autoregressive transformers
Sotiris Anagnostidis, Dario Pavllo, Luca Biggio, Lorenzo Noci, Aurélien Lucchi, and Thomas Hofmann · 2023
Cited alongside, same era.
Adapting language models to compress contexts
Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen · 2023
Cited alongside, same era.
LLMLingua: Compressing prompts for accelerated inference of large language models
Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu · 2023
Cited alongside, same era.
Solar 10.7b: Scaling large language models with simple yet effective depth up-scaling, 2023
Dahyun Kim, Chanjun Park, Sanghoon Kim, Wonsung Lee, Wonho Song, Yunsu Kim, Hyeonwoo Kim, Yungi Kim, Hyeonju Lee, Jihoo Kim, Changbae Ahn, Seonghoon Yang, Sukyung Lee, Hyunbyung Park, Gyoungjin Gim, Mikyoung Cha, Hwalsuk Lee, and Sunghun Kim · 2023
Flashattention-2: Faster attention with better parallelism and work partitioning
Tri Dao · 2024
Later among the works it cites.
SyllabusQA: A course logistics question answering dataset
Nigel Fernandez, Alexander Scarlatos, and Andrew Lan · 2024
Later among the works it cites.
In-context autoencoder for context compression in a large language model
Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei · 2024
Later among the works it cites.
Ragged: Towards informed design of retrieval augmented generation systems
Jennifer Hsia, Afreen Shaikh, Zhiruo Wang, and Graham Neubig · 2024
Later among the works it cites.
DSLR: Document refinement with sentence-level re-ranking and reconstruction to enhance retrieval-augmented generation
Taeho Hwang, Soyeong Jeong, Sukmin Cho, SeungYoon Han, and Jong Park · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Efficient memory management for large language model serving with pagedattention, 2023
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica · 2023
Cited alongside, same era.
Naver labs europe (splade) @ trec deep learning 2022, 2023
Carlos Lassance and Stéphane Clinchant · 2023
Cited alongside, same era.
Compressing context to enhance inference efficiency of large language models
Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin · 2023
Cited alongside, same era.
When not to trust language models: Investigating effectiveness of parametric and non-parametric memories
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi · 2023
Cited alongside, same era.
When not to trust language models: Investigating effectiveness of parametric and non-parametric memories
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi · 2023
Cited alongside, same era.
Factscore: Fine-grained atomic evaluation of factual precision in long form text generation, 2023
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi · 2023
Cited alongside, same era.
Overview of BioASQ 2023: The Eleventh BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering , pp. 227–250
Anastasios Nentidis, Georgios Katsimpras, Anastasia Krithara, Salvador Lima-López, Eulalia Farré-Maduell, Luis Gasco Sanchez, Martin Krallinger, and Georgios Paliouras · 2023
Cited alongside, same era.
LongLLMLingua: Accelerating and enhancing LLMs in long context scenarios via prompt compression
Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu · 2024
Later among the works it cites.
Splade-v3: New baselines for splade, 2024
Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant · 2024
Later among the works it cites.
RA-DIT: Retrieval-augmented dual instruction tuning
Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Richard James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, Luke Zettlemoyer, and Wen tau Yih · 2024
Later among the works it cites.
Dynamic memory compression: Retrofitting llms for accelerated inference
Piotr Nawrot, Adrian La’ncucki, Marcin Chochowski, David Tarjan, and E. Ponti · 2024
Later among the works it cites.
LLMLingua-2: Data distillation for efficient and faithful task-agnostic prompt compression
Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Ruhle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang · 2024
Later among the works it cites.
BERGEN: A benchmarking library for retrieval-augmented generation
David Rau, Hervé Déjean, Nadezhda Chirkova, Thibault Formal, Shuai Wang, Stéphane Clinchant, and Vassilina Nikoulina · 2024
Later among the works it cites.
RECOMP: Improving retrieval-augmented LMs with context compression and selective augmentation
Fangyuan Xu, Weijia Shi, and Eunsol Choi · 2024
Later among the works it cites.
Compact: Compressing retrieved documents actively for question answering, 2024
Chanwoong Yoon, Taewhoo Lee, Hyeon Hwang, Minbyul Jeong, and Jaewoo Kang · 2024
Later among the works it cites.
Making retrieval-augmented language models robust to irrelevant context
Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant · 2024
Later among the works it cites.
Accelerating inference of retrieval-augmented generation via sparse context selection
Yun Zhu, Jia-Chen Gu, Caitlin Sikora, Ho Ko, Yinxiao Liu, Chu-Cheng Lin, Lei Shu, Liangchen Luo, Lei Meng, Bang Liu, and Jindong Chen · 2024
Later among the works it cites.
Pisco: Pretty simple compression for retrieval-augmented generation
Maxime Louis, Hervé Déjean, and Stéphane Clinchant · 2025
Closest in time.