ReLearn: Unlearning via Learning for Large Language Models

Sentence-bert: Sentence embeddings using siamese bert-networks

Original

Nils Reimers and Iryna Gurevych. 2019 · 1908

Earlier work this paper cites.

Vocabulaire de Jean Giraudoux: structure et évolution: statistique et informatique appliquées à l’étude des textes à partir des données du Trésor de la langue

Etienne Brunet. 1978 · 1978

Earlier work this paper cites.

Some simple measures of richness of vocabulary

A. Honoré. 1979 · 1979

Earlier work this paper cites.

Extracting training data from large language models

Original

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. 2021 · 2012

Earlier work this paper cites.

Memory loss in Alzheimer’s disease

H. Jahn. 2013 · 2013

Earlier work this paper cites.

General data protection regulation (gdpr)

European Union. 2016 · 2016

Earlier work this paper cites.

Linguistic features identify alzheimer’s disease in narrative speech

K. C. Fraser, J. A. Meltzer, F. Rudzicz, and P. Garrard. 2016 · 2016

Earlier work this paper cites.

An update on memory reconsolidation updating

Jonathan L. C. Lee, Karim Nader, and Daniela Schiller. 2017 · 2017

Earlier work this paper cites.

WikiQA: A challenge dataset for open-domain question answering

Yi Yang, Wen-tau Yih, and Christopher Meek. 2015 · 2018

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021

Earlier work this paper cites.

Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space

Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. 2022 · 2022

Earlier work this paper cites.

Knowledge unlearning for mitigating privacy risks in language models

Original

Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2022 · 2022

Earlier work this paper cites.

Prosocialdialog: A prosocial backbone for conversational agents

Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. 2022 · 2022

Earlier work this paper cites.

Quark: Controllable text generation with reinforced unlearning

Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022 · 2022

Earlier work this paper cites.

Eliciting latent predictions from transformers with the tuned lens

Original

Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023 · 2023

Earlier work this paper cites.

Unlearn what you want to forget: Efficient unlearning for llms

Original

Jiaao Chen and Diyi Yang. 2023 · 2023

Earlier work this paper cites.

Who’s harry potter? approximate unlearning in llms

Original

Ronen Eldan and Mark Russinovich. 2023 · 2023

Earlier work this paper cites.

Dissecting recall of factual associations in auto-regressive language models

Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023 · 2023

Earlier work this paper cites.

Knowledge unlearning for mitigating privacy risks in language models

Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2023 · 2023

Earlier work this paper cites.

A watermark for large language models

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023 · 2023

Earlier work this paper cites.

Recent advances in natural language processing via large pre-trained language models: A survey

Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. 2023 · 2023

Earlier work this paper cites.

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023 · 2023

Earlier work this paper cites.

tasksource: Structured dataset preprocessing annotations for frictionless extreme multi-task learning and evaluation

Original

Damien Sileo. 2023 · 2023

Cited alongside, same era.

Evaluation metrics in the era of GPT-4: Reliably evaluating large language models on sequence to sequence tasks

Andrea Sottana, Bin Liang, Kai Zou, and Zheng Yuan. 2023 · 2023

Cited alongside, same era.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023 · 2023

Cited alongside, same era.

Dr.icl: Demonstration-retrieved in-context learning

Luo Man, Xu Xin, Dai Zhuyun, Pasupat Panupong, Kazemi Mehran, Baral Chitta, Imbrasaite Vaiva, and Y Zhao Vincent. 2024 · 2024

Later among the works it cites.

Gpt-4o system card

Original

OpenAI, :, Aaron Hurst, et al. 2024 · 2024

Later among the works it cites.

In-context unlearning: Language models as few shot unlearners

Original

Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2024 · 2024

Later among the works it cites.

Extracting unlearned information from llms with activation steering

Original

Atakan Seyitoğlu, Aleksei Kuvshinov, Leo Schwinn, and Stephan Günnemann. 2024 · 2024

Later among the works it cites.

Muse: Machine unlearning six-way evaluation for language models

Original