Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods

Measuring massive multitask language understanding

Original

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2009

Earlier work this paper cites.

Ethical and social risks of harm from language models

Original

Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021 · 2021

Earlier work this paper cites.

Foundation models and fair use

Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A. Lemley, and Percy Liang. 2023 · 2023

Earlier work this paper cites.

Zephyr: Direct distillation of lm alignment

Original

Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. 2023 · 2023

Earlier work this paper cites.

Large language model unlearning

Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2023 · 2023

Cited alongside, same era.

The llama 3 herd of models

Original

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024 · 2024

Cited alongside, same era.

Rethinking machine unlearning for large language models

Original

Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R Varshney, et al. 2024 · 2024

Cited alongside, same era.

Eight methods to evaluate robust unlearning in llms

Original

Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menell. 2024 · 2024

Cited alongside, same era.

The wmdp benchmark: Measuring and reducing malicious use with unlearning

Original

Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. 2024a

Negative preference optimization: From catastrophic collapse to effective unlearning

Original

Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024 · 2024

Closest in time.

Judging llm-as-a-judge with mt-bench and chatbot arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, et al. 2024 · 2024

Closest in time.

An adversarial perspective on machine unlearning for ai safety

Original

Jakub Łucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. 2024 · 2024

Closest in time.