Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models

Deep Learning Recommendation Model for Personalization and Recommendation Systems

Original

Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, and Misha Smelyanskiy. 2019 · 1906

Earlier work this paper cites.

Okapi at TREC-3. In Text Retrieval Conference

Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994 · 1994

Earlier work this paper cites.

Generation and evaluation of user tailored responses in multimodal dialogue

M.A. Walker, S.J. Whittaker, A. Stent, P. Maloor, J. Moore, M. Johnston, and G. Vasireddy. 2004 · 2003

Earlier work this paper cites.

ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out . Association for Computational Linguistics, Barcelona, Spain, 74–81

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Personalized recommendation in social tagging systems using hierarchical clustering. In Proceedings of the 2008 ACM Conference on Recommender Systems (Lausanne, Switzerland) (RecSys ’08) . Association for Computing Machinery, New York, NY, USA, 259–266

Andriy Shepitsen, Jonathan Gemmell, Bamshad Mobasher, and Robin Burke. 2008 · 2008

Earlier work this paper cites.

User Language Model for Collaborative Personalized Search

Gui-Rong Xue, Jie Han, Yong Yu, and Qiang Yang. 2009 · 2009

Earlier work this paper cites.

Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014

Earlier work this paper cites.

Effects of Language Modeling and Its Personalization on Touchscreen Typing Performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15) . Association for Computing Machinery, New York, NY, USA, 649–658

Andrew Fowler, Kurt Partridge, Ciprian Chelba, Xiaojun Bi, Tom Ouyang, and Shumin Zhai. 2015 · 2015

Earlier work this paper cites.

Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR) . San Diega, CA, USA

Diederik Kingma and Jimmy Ba. 2015 · 2015

Earlier work this paper cites.

Avocado Research Email Collection

Oard, Douglas, Webber, William, Kirsch, David A., and Golitsynskiy, Sergey. 2015 · 2015

Earlier work this paper cites.

Personalized Recommendation via Parameter-Free Contextual Bandits. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (Santiago, Chile) (SIGIR ’15) . Association for Computing Machinery, New York, NY, USA, 323–332

Liang Tang, Yexi Jiang, Lei Li, Chunqiu Zeng, and Tao Li. 2015 · 2015

Earlier work this paper cites.

Beam Search Strategies for Neural Machine Translation. In Proceedings of the First Workshop on Neural Machine Translation , Thang Luong, Alexandra Birch, Graham Neubig, and Andrew Finch (Eds.). Association for Computational Linguistics, Vancouver, 56–60

Markus Freitag and Yaser Al-Onaizan. 2017 · 2017

Earlier work this paper cites.

Attention is All you Need. In Advances in Neural Information Processing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Parameter-Efficient Transfer Learning for NLP. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97) , Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 2790–2799

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019 · 2019

Earlier work this paper cites.

The Personalization of Conversational Agents in Health Care: Systematic Review

Ahmet Baki Kocaballi, Shlomo Berkovsky, Juan C Quiroz, Liliana Laranjo, Huong Ly Tong, Dana Rezazadegan, Agustina Briatore, and Enrico Coiera. 2019 · 2019

Earlier work this paper cites.

Personalization and the Conversational Web. In Web Information Systems and Technologies , María José Escalona, Francisco Domínguez Mayo, Tim A. Majchrzak, and Valérie Monfort (Eds.). Springer International Publishing, Cham, 56–77

Konstantinos N. Vavliakis, Maria Th. Kotouza, Andreas L. Symeonidis, and Pericles A. Mitkas. 2019 · 2019

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems (<conf-loc>, <city>Vancouver</city>, <state>BC</state>, <country>Canada</country>, </conf-loc>) (NIPS ’20) . Curran Associates Inc., Red Hook, NY, USA, Article 793, 16 pages

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020 · 2020

Earlier work this paper cites.

Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , Qun Liu and David Schlangen (Eds.). Association for Computational Linguistics, Online, 38–45

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020 · 2020

Earlier work this paper cites.

Personalized Search-based Query Rewrite System for Conversational AI. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI , Alexandros Papangelis, Paweł Budzianowski, Bing Liu, Elnaz Nouri, Abhinav Rastogi, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Online, 179–188

Eunah Cho, Ziyan Jiang, Jie Hao, Zheng Chen, Saurabh Gupta, Xing Fan, and Chenlei Guo. 2021 · 2021

Cited alongside, same era.

AOL4PS: A Large-scale Data Set for Personalized Search

Qian Guo, Wei Chen, and Huaiyu Wan. 2021 · 2021

Cited alongside, same era.

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (Eds.). Association for Computational Linguistics, Online, 874–880

Gautier Izacard and Edouard Grave. 2021 · 2021

Cited alongside, same era.

Pre-Training Multi-Modal Dense Retrievers for Outside-Knowledge Visual Question Answering. In Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval (<conf-loc>, <city>Taipei</city>, <country>Taiwan</country>, </conf-loc>) (ICTIR ’23) . Association for Computing Machinery, New York, NY, USA, 169–176

Alireza Salemi, Mahta Rafiee, and Hamed Zamani. 2023b · 2023

Later among the works it cites.

Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering

Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023 · 2023

Later among the works it cites.

LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning

Original

Longteng Zhang, Lin Zhang, Shaohuai Shi, Xiaowen Chu, and Bo Li. 2023 · 2023

Later among the works it cites.