Fetching the paper…
Reading the bibliography…
Retrieval-augmented generation (RAG) can enhance the generation quality of large language models (LLMs) by incorporating external token databases.
Video google: A text retrieval approach to object matching in videos
Sivic, J. and Zisserman, A · 2003
Earlier work this paper cites.
Product quantization for nearest neighbor search
Jegou, H., Douze, M., and Schmid, C · 2010
Earlier work this paper cites.
Learning deep structured semantic models for web search using clickthrough data
Huang, P.-S., He, X., Gao, J., Deng, L., Acero, A., and Heck, L · 2013
Earlier work this paper cites.
Approximate nearest neighbor algorithm based on navigable small world graphs
Malkov, Y., Ponomarenko, A., Logvinov, A., and Krylov, V · 2014
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs
Malkov, Y. A. and Yashunin, D. A · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al · 2018
Earlier work this paper cites.
Deeper text understanding for ir with contextual neural language modeling
Dai, Z. and Callan, J · 2019
Earlier work this paper cites.
Billion-scale similarity search with gpus
Johnson, J., Douze, M., and Jégou, H · 2019
Earlier work this paper cites.
Generalization through memorization: Nearest neighbor language models
Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., and Lewis, M · 2019
Earlier work this paper cites.
Cedr: Contextualized embeddings for document ranking
MacAvaney, S., Yates, A., Cohan, A., and Goharian, N · 2019
Earlier work this paper cites.
Nogueira, R. and Cho, K · 2019
Earlier work this paper cites.
Sentence-bert: Sentence embeddings using siamese bert-networks
Reimers, N. and Gurevych, I · 2019
Earlier work this paper cites.
Longformer: The long-document transformer
Beltagy, I., Peters, M. E., and Cohan, A · 2020
Cited alongside, same era.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Cited alongside, same era.
Retrieval augmented language model pre-training
Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M · 2020
Cited alongside, same era.
Leveraging passage retrieval with generative models for open domain question answering
Izacard, G. and Grave, E · 2020
Cited alongside, same era.
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t · 2020
Cited alongside, same era.
Colbertv2: Effective and efficient retrieval via lightweight late interaction
Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., and Zaharia, M · 2021
Later among the works it cites.
Improving language models by retrieving from trillions of tokens
Borgeaud, S., Mensch, A., Hoffmann, J., Cai, T., Rutherford, E., Millican, K., Van Den Driessche, G. B., Lespiau, J.-B., Damoc, B., Clark, A., et al · 2022
Later among the works it cites.
Glam: Efficient scaling of language models with mixture-of-experts
Du, N., Huang, Y., Dai, A. M., Tong, S., Lepikhin, D., Xu, Y., Krikun, M., Zhou, Y., Yu, A. W., Firat, O., et al · 2022
Later among the works it cites.
Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity
Fedus, W., Zoph, B., and Shazeer, N · 2022
Later among the works it cites.
Gptq: Accurate post-training quantization for generative pre-trained transformers
Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Nearest neighbor machine translation
Khandelwal, U., Fan, A., Jurafsky, D., Zettlemoyer, L., and Lewis, M · 2020
Cited alongside, same era.
Colbert: Efficient and effective passage search via contextualized late interaction over bert
Khattab, O. and Zaharia, M · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2020
Cited alongside, same era.
Documenting large webtext corpora: A case study on the colossal clean crawled corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., and Gardner, M · 2021
Cited alongside, same era.
Internet-augmented dialogue generation
Komeili, M., Shuster, K., and Weston, J · 2021
Cited alongside, same era.
Fast nearest neighbor machine translation
Meng, Y., Li, X., Zheng, X., Wu, F., Sun, X., Zhang, T., and Li, J · 2021
Cited alongside, same era.
End-to-end training of neural retrievers for open-domain question answering
Sachan, D. S., Patwary, M., Shoeybi, M., Kant, N., Ping, W., Hamilton, W. L., and Catanzaro, B · 2021
Cited alongside, same era.
Later among the works it cites.
Few-shot learning with retrieval augmented language models
Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., and Grave, E · 2022
Later among the works it cites.
Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions
Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A · 2022
Later among the works it cites.
Surface-based retrieval reduces perplexity of retrieval-augmented language models
Doostmohammadi, E., Norlund, T., Kuhlmann, M., and Johansson, R · 2023
Later among the works it cites.
On the generalization ability of retrieval-enhanced transformers
Norlund, T., Doostmohammadi, E., Johansson, R., and Kuhlmann, M · 2023
Later among the works it cites.
In-context retrieval-augmented language models
Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D., Shashua, A., Leyton-Brown, K., and Shoham, Y · 2023
Later among the works it cites.
Knn-lm does not improve open-ended text generation
Wang, S., Song, Y., Drozdov, A., Garimella, A., Manjunatha, V., and Iyyer, M · 2023
Later among the works it cites.
Why do nearest neighbor language models work?
Xu, F. F., Alon, U., and Neubig, G · 2023
Later among the works it cites.