Entropy-Based Decoding for Retrieval-Augmented Large Language Models

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020 · 1901

Earlier work this paper cites.

Training products of experts by minimizing contrastive divergence

Geoffrey E Hinton. 2002 · 2002

Earlier work this paper cites.

Leveraging passage retrieval with generative models for open domain question answering

Original

Gautier Izacard and Edouard Grave. 2020 · 2007

Earlier work this paper cites.

Kilt: a benchmark for knowledge intensive language tasks

Original

Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, et al. 2020 · 2009

Earlier work this paper cites.

Semantic parsing on freebase from question-answer pairs

Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013 · 2013

Earlier work this paper cites.

A diversity-promoting objective function for neural conversation models

Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016 · 2016

Earlier work this paper cites.

Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension

Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017 · 2017

Earlier work this paper cites.

Natural questions: a benchmark for question answering research

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019 · 2019

Earlier work this paper cites.

Latent retrieval for weakly supervised open domain question answering

Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Retrieval augmented language model pre-training

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020 · 2020

Earlier work this paper cites.

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020 · 2020

Earlier work this paper cites.

Dexperts: Decoding-time controlled text generation with experts and anti-experts

Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021 · 2021

Earlier work this paper cites.

Entity-based knowledge conflicts in question answering

Original

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021 · 2021

Cited alongside, same era.

Improving language models by retrieving from trillions of tokens

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022 · 2022

Cited alongside, same era.

Unsupervised dense information retrieval with contrastive learning

Izacard Gautier, Caron Mathilde, Hosseini Lucas, Riedel Sebastian, Bojanowski Piotr, Joulin Armand, and Grave Edouard. 2022 · 2022

Cited alongside, same era.

Contrastive decoding: Open-ended text generation as optimization

Original

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2022 · 2022

Cited alongside, same era.

Training language models to follow instructions with human feedback

Contrastive decoding improves reasoning in large language models

Original

Sean O’Brien and Mike Lewis. 2023 · 2023

Later among the works it cites.

In-context retrieval-augmented language models

Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023 · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Later among the works it cites.

A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation

Original

Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. 2023 · 2023

Later among the works it cites.

Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts