2024

Personalized Large Language Models

Woźniak, Stanisław, Koptyra, Bartłomiej, Janz, Arkadiusz et al.

Understand

Large language models (LLMs) have significantly advanced Natural Language Processing (NLP) tasks in recent years.

However, their universal nature poses limitations in scenarios requiring personalized responses, such as recommendation systems and chatbots.
This paper investigates methods to personalize LLMs, comparing fine-tuning and zero-shot reasoning approaches on subjective tasks.
Results demonstrate that personalized fine-tuning improves model reasoning compared to non-personalized models.

Built on

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 1877–1901
1901
Earlier work this paper cites.
R. M. French, “Catastrophic forgetting in connectionist networks,” Trends in cognitive sciences , vol. 3, no. 4, pp. 128–135, 1999
1999
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of naacL-HLT , vol. 1. Minneapolis, Minnesota, 2019, p. 2
2019
Earlier work this paper cites.
D. Demszky, D. Movshovitz-Attias, J. Ko, A. Cowen, G. Nemade, and S. Ravi, “GoEmotions: A dataset of fine-grained emotions,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, Eds. Online: Association for Computational Linguistics, Jul. 2020
2020
Earlier work this paper cites.
I. Price, J. Gifford-Moore, J. Flemming, S. Musker, M. Roichman, G. Sylvain, N. Thain, L. Dixon, and J. Sorensen, “Six attributes of unhealthy conversations,” in Proceedings of the Fourth Workshop on Online Abuse and Harms , S. Akiwowo, B. Vidgen, V. Prabhakaran, and Z. Waseem, Eds. Online: Association for Computational Linguistics, Nov. 2020
2020
Earlier work this paper cites.
M. Wierzba, M. Riegel, J. Kocoń, P. Miłkowski, A. Janz, K. Klessa, K. Juszczyk, B. Konat, D. Grimling, M. Piasecki et al. , “Emotion norms for 6000 polish word meanings with a direct mapping to the polish wordnet,” Behavior Research Methods , pp. 1–16, 2021
2021
Earlier work this paper cites.
J. Kocoń, A. Figas, M. Gruza, D. Puchalska, T. Kajdanowicz, and P. Kazienko, “Offensive, aggressive, and hate speech analysis: From data-centric to human-centered approach,” Information Processing & Management , vol. 58, no. 5, p. 102643, 2021
2021
Earlier work this paper cites.
K. Kanclerz, A. Figas, M. Gruza, T. Kajdanowicz, J. Kocoń, D. Puchalska, and P. Kazienko, “Controversy and conformity: from generalized to personalized aggressiveness detection,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , 2021, pp. 5915–5926
2021
Earlier work this paper cites.
J. Kocoń, M. Gruza, J. Bielaniewicz, D. Grimling, K. Kanclerz, P. Miłkowski, and P. Kazienko, “Learning personal human biases and representations for subjective tasks in natural language processing,” in 2021 IEEE International Conference on Data Mining (ICDM) . IEEE, 2021, pp. 1168–1173
2021
Earlier work this paper cites.
P. Miłkowski, M. Gruza, K. Kanclerz, P. Kazienko, D. Grimling, and J. Kocoń, “Personal bias in prediction of emotions elicited by textual opinions,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop , 2021, pp. 248–259
2021
Earlier work this paper cites.
T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” Advances in neural information processing systems , vol. 35, pp. 22 199–22 213, 2022
2022
Earlier work this paper cites.
Q. Dong, L. Li, D. Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, J. Xu, and Z. Sui, “A survey for in-context learning,” arXiv preprint arXiv:2301.00234 , 2022
Original
2022
Earlier work this paper cites.
J. Bielaniewicz, K. Kanclerz, P. Miłkowski, M. Gruza, K. Karanowski, P. Kazienko, and J. Kocoń, “Deep-sheep: Sense of humor extraction from embeddings in the personalized context,” in 2022 IEEE International Conference on Data Mining Workshops (ICDMW) . IEEE, 2022, pp. 967–974
2022
Earlier work this paper cites.
A. Ngo, A. Candri, T. Ferdinan, J. Kocoń, and W. Korczynski, “Studemo: A non-aggregated review dataset for personalized emotion recognition,” in Proceedings of the 1st Workshop on Perspectivist Approaches to NLP@ LREC2022 , 2022, pp. 46–55
2022
Earlier work this paper cites.
K. Kanclerz, M. Gruza, K. Karanowski, J. Bielaniewicz, P. Miłkowski, J. Kocoń, and P. Kazienko, “What if ground truth is subjective? personalized deep neural hate speech detection,” in Proceedings of the 1st Workshop on Perspectivist Approaches to NLP@ LREC2022 , 2022, pp. 37–45
2022
Earlier work this paper cites.
P. Miłkowski, S. Saganowski, M. Gruza, P. Kazienko, M. Piasecki, and J. Kocoń, “Multitask personalized recognition of emotions evoked by textual content,” in 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) . IEEE, 2022, pp. 347–352
2022
Earlier work this paper cites.

Similar

F. Mireshghallah, V. Shrivastava, M. Shokouhi, T. Berg-Kirkpatrick, R. Sim, and D. Dimitriadis, “Useridentifier: Implicit user representations for simple and effective personalized sentiment analysis,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2022, pp. 3449–3456
2022
Cited alongside, same era.
H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma et al. , “Scaling instruction-finetuned language models,” arXiv preprint arXiv:2210.11416 , 2022
Original
2022
Cited alongside, same era.
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang et al. , “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems , vol. 35, pp. 27 730–27 744, 2022
2022
Cited alongside, same era.
S. Black, S. Biderman, E. Hallahan, Q. Anthony, L. Gao, L. Golding, H. He, C. Leahy, K. McDonell, J. Phang et al. , “Gpt-neox-20b: An open-source autoregressive language model,” arXiv preprint arXiv:2204.06745 , 2022
Original
2022
Cited alongside, same era.
J. Kocoń, “Deep emotions across languages: A novel approach for sentiment propagation in multilingual wordnets,” in 2023 IEEE International Conference on Data Mining Workshops (ICDMW) . IEEE, 2023, pp. 744–749
2023
Cited alongside, same era.
S. Porsdam Mann, B. D. Earp, N. Møller, S. Vynn, and J. Savulescu, “Autogen: A personalized large language model for academic enhancement—ethics and proof of principle,” The American Journal of Bioethics , vol. 23, no. 10, pp. 28–41, 2023
2023
Cited alongside, same era.
H. Lyu, S. Jiang, H. Zeng, Y. Xia, and J. Luo, “Llm-rec: Personalized recommendation via prompting large language models,” arXiv preprint arXiv:2307.15780 , 2023
Original
2023
Cited alongside, same era.
M. Abbasian, I. Azimi, A. M. Rahmani, and R. Jain, “Conversational health agents: A personalized llm-powered agent framework,” arXiv preprint arXiv:2310.02374 , 2023
Original
2023
Cited alongside, same era.
Z. Chen, “Palr: Personalization aware llms for recommendation,” arXiv preprint arXiv:2305.07622 , 2023
Original
2023
Cited alongside, same era.
J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, “Tidybot: Personalized robot assistance with large language models,” arXiv preprint arXiv:2305.05658 , 2023
Original
2023
Cited alongside, same era.
P. Miłkowski, K. Karanowski, P. Wielopolski, J. Kocoń, P. Kazienko, and M. Zięba, “Modeling uncertainty in personalized emotion prediction with normalizing flows,” in 2023 IEEE International Conference on Data Mining Workshops (ICDMW) . IEEE, 2023
2023
Cited alongside, same era.
K. Kanclerz, J. Bielaniewicz, M. Gruza, J. Kocoń, S. Woźniak, and P. Kazienko, “Towards model-based data acquisition for subjective multi-task nlp problems,” in 2023 IEEE International Conference on Data Mining Workshops (ICDMW) . IEEE, 2023, pp. 726–735
2023
Cited alongside, same era.

Then

T. Ferdinan and J. Kocoń, “Personalized models resistant to malicious attacks for human-centered trusted ai,” in The AAAI-23 Workshop on Artificial Intelligence Safety (SafeAI 2023) , 2023
2023
Later among the works it cites.
P. Kazienko, J. Bielaniewicz, M. Gruza, K. Kanclerz, K. Karanowski, P. Miłkowski, and J. Kocoń, “Human-centered neural reasoning for subjective content processing: Hate speech, emotions, and humor,” Information Fusion , vol. 94, pp. 43–65, 2023
2023
Later among the works it cites.
J. Kocoń, I. Cichecki, O. Kaszyca, M. Kochanek, D. Szydło, J. Baran et al. , “Chatgpt: Jack of all trades, master of none,” Information Fusion , p. 101861, 2023
2023
Later among the works it cites.
S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. R. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023
2023
Later among the works it cites.
S. Kotha, J. M. Springer, and A. Raghunathan, “Understanding catastrophic forgetting in language models via implicit inference,” arXiv preprint arXiv:2309.10105 , 2023
Original
2023
Later among the works it cites.
Y. Zhai, S. Tong, X. Li, M. Cai, Q. Qu, Y. J. Lee, and Y. Ma, “Investigating the catastrophic forgetting in multimodal large language models,” ArXiv , vol. abs/2309.10313, 2023
Original
2023
Later among the works it cites.
A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier et al. , “Mistral 7b,” arXiv preprint arXiv:2310.06825 , 2023
Original
2023
Later among the works it cites.
B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin et al. , “Code llama: Open foundation models for code,” arXiv preprint arXiv:2308.12950 , 2023
Original
2023
Later among the works it cites.
H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288 , 2023
Original
2023
Later among the works it cites.
Y. Li, S. Bubeck, R. Eldan, A. Del Giorno, S. Gunasekar, and Y. T. Lee, “Textbooks are all you need ii: phi-1.5 technical report,” arXiv preprint arXiv:2309.05463 , 2023
Original
2023
Later among the works it cites.
T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient finetuning of quantized LLMs,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023
2023
Later among the works it cites.
R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/forum?id=HPuSIXJaa9
2023
Later among the works it cites.
E. Cambria, X. Zhang, R. Mao, M. Chen, and K. Kwok, “Senticnet 8: Fusing emotion ai and commonsense ai for interpretable, trustworthy, and explainable affective computing,” in International Conference on Human-Computer Interaction (HCII) , 2024
2024
Closest in time.
L. Zhu, R. Mao, E. Cambria, and B. J. Jansen, “Neurosymbolic ai for personalized sentiment analysis,” in Proceedings of HCII , 2024
2024
Closest in time.
——, “Fortifying nlp models against poisoning attacks: The power of personalized prediction architectures,” Information Fusion , p. 102692, 2024
2024
Closest in time.
C.-A. Li and H.-Y. Lee, “Examining forgetting in continual pre-training of aligned large language models,” arXiv preprint arXiv:2401.03129 , 2024
Original
2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…