Syntriever: How to Train Your Retriever with Synthetic Data from LLMs

Fine-tuning language models from human preferences

Original

Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019 · 1909

Earlier work this paper cites.

Rank analysis of incomplete block designs: I. the method of paired comparisons

Ralph Allan Bradley and Milton E Terry. 1952 · 1952

Earlier work this paper cites.

Individual choice behavior , volume 4

R Duncan Luce. 1959 · 1959

Earlier work this paper cites.

The analysis of permutations

Robin L Plackett. 1975 · 1975

Earlier work this paper cites.

The probabilistic relevance framework: Bm25 and beyond

Stephen Robertson, Hugo Zaragoza, et al. 2009 · 2009

Earlier work this paper cites.

Adam: A method for stochastic optimization

Original

Diederik P Kingma. 2014 · 2014

Earlier work this paper cites.

Ms marco: A human-generated machine reading comprehension dataset

Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016 · 2016

Earlier work this paper cites.

Reading wikipedia to answer open-domain questions

Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017 · 2017

Earlier work this paper cites.

Bert: Pre-training of deep bidirectional transformers for language understanding

Original

Jacob Devlin. 2018 · 2018

Earlier work this paper cites.

Representation learning with contrastive predictive coding

Original

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018 · 2018

Earlier work this paper cites.

Natural questions: a benchmark for question answering research

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019 · 2019

Earlier work this paper cites.

Latent retrieval for weakly supervised open domain question answering

Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Sentence-bert: Sentence embeddings using siamese bert-networks

Nils Reimers and Iryna Gurevych. 2019 · 2019

Earlier work this paper cites.

Analyzing and improving representations with the soft nearest neighbor loss

Nicholas Frosst, Nicolas Papernot, and Geoffrey Hinton. 2019 · 2020

Earlier work this paper cites.

Retrieval augmented language model pre-training

Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020 · 2020

Earlier work this paper cites.

Momentum contrast for unsupervised visual representation learning

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020 · 2020

Earlier work this paper cites.

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020 · 2020

Cited alongside, same era.

Neural text degeneration with unlikelihood training

Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020 · 2020

Cited alongside, same era.

Unsupervised dense information retrieval with contrastive learning

Original

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021 · 2021

Cited alongside, same era.

Rocketqa: An optimized training approach to dense passage retrieval for open-domain question answering

Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2021 · 2021

Cited alongside, same era.

Rocketqav2: A joint training method for dense passage retrieval and passage re-ranking

Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, Qiaoqiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021 · 2021

Textbooks are all you need

Original

Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. 2023 · 2023

Later among the works it cites.

Mistral 7b

Original

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023

Later among the works it cites.

Textbooks are all you need ii: phi-1.5 technical report

Original

Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023 · 2023

Later among the works it cites.

Code llama: Open foundation models for code

Original

Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023 · 2023

Later among the works it cites.