A Tale of Trust and Accuracy: Base vs. Instruct LLMs in RAG Systems

Fast transformer decoding: One write-head is all you need

Original

Noam Shazeer. 2019 · 1911

Earlier work this paper cites.

Leveraging passage retrieval with generative models for open domain question answering

Original

Gautier Izacard and Edouard Grave. 2021 · 2007

Earlier work this paper cites.

TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension

Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017 · 2017

Earlier work this paper cites.

Improving language understanding by generative pre-training

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018 · 2018

Earlier work this paper cites.

The secret sharer: evaluating and testing unintended memorization in neural networks

Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019 · 2019

Earlier work this paper cites.

Billion-scale similarity search with GPUs

Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019 · 2019

Earlier work this paper cites.

Natural questions: A benchmark for question answering research

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019 · 2019

Earlier work this paper cites.

Latent retrieval for weakly supervised open domain question answering

Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

A discrete hard EM approach for weakly supervised question answering

Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019 · 2019

Earlier work this paper cites.

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020 · 2020

Earlier work this paper cites.

Unsupervised dense information retrieval with contrastive learning

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021 · 2021

Earlier work this paper cites.

Hurdles to progress in long-form question answering

Kalpesh Krishna, Aurko Roy, and Mohit Iyyer. 2021 · 2021

Earlier work this paper cites.

Entity-based knowledge conflicts in question answering

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021 · 2021

Earlier work this paper cites.

Retrieval augmentation reduces hallucination in conversation

Original

Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021 · 2021

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Original

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022 · 2022

Earlier work this paper cites.

Robust policy optimization in deep reinforcement learning

Original

Md Masudur Rahman and Yexiang Xue. 2022 · 2022

Earlier work this paper cites.

Measure and improve robustness in nlp models: A survey

Original

Xuezhi Wang, Haohan Wang, and Diyi Yang. 2022 · 2022

Cited alongside, same era.

GQA: Training generalized multi-query transformer models from multi-head checkpoints

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. 2023 · 2023

Cited alongside, same era.

The falcon series of open language models

Original

Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023 · 2023

Cited alongside, same era.

Self-rag: Learning to retrieve, generate, and critique through self-reflection

Original

Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023 · 2023

Cited alongside, same era.

Siren’s song in the ai ocean: A survey on hallucination in large language models

Original

Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023 · 2023

Later among the works it cites.

Chatgpt hallucinates when attributing answers

Original

Guido Zuccon, Bevan Koopman, and Razia Shaik. 2023 · 2023

Later among the works it cites.

Benchmarking large language models in retrieval-augmented generation

Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024 · 2024

Closest in time.

The power of noise: Redefining retrieval for rag systems

Original

Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024 · 2024

Closest in time.

The faiss library