Fetching the paper…

Reading the bibliography…

2024

Position: LLM Unlearning Benchmarks are Weak Measures of Progress

Thaker, Pratiksha, Hu, Shengyuan, Kale, Neil et al.

Original
PDF
Post on X

Understand

Unlearning methods have the potential to improve the privacy and safety of large language models (LLMs) by removing sensitive or harmful information post hoc.

The LLM unlearning research community has increasingly turned toward empirical benchmarks to assess the effectiveness of such methods.
In this paper, we find that existing benchmarks provide an overly optimistic and potentially misleading view on the effectiveness of candidate unlearning methods.
By introducing simple, benign modifications to a number of popular benchmarks, we expose instances where supposedly unlearned information remains accessible, or where the unlearning process has degraded the model's performance on retained information to a much greater extent than indicated by the original benchmark.

Position: LLM Unlearning Benchmarks are Weak Measures of Progress · Around

Built on

C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out , 2004, pp. 74–81
2004
Earlier work this paper cites.
C. Dwork, “Differential privacy,” in International colloquium on automata, languages, and programming . Springer, 2006, pp. 1–12
2006
Earlier work this paper cites.
Y. Cao and J. Yang, “Towards making systems forget with machine unlearning,” in 2015 IEEE symposium on security and privacy . IEEE, 2015, pp. 463–480
2015
Earlier work this paper cites.
T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” arXiv preprint arXiv:1809.02789 , 2018
Original
2018
Earlier work this paper cites.
A. Ginart, M. Guan, G. Valiant, and J. Y. Zou, “Making ai forget you: Data deletion in machine learning,” Advances in neural information processing systems , vol. 32, 2019
2019
Earlier work this paper cites.
S. Shintre, K. A. Roundy, and J. Dhaliwal, “Making machine learning forget,” in Privacy Technologies and Policy: 7th Annual Privacy Forum, APF 2019, Rome, Italy, June 13–14, 2019, Proceedings 7 . Springer, 2019, pp. 72–83
2019
Earlier work this paper cites.
R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi, “Hellaswag: Can a machine really finish your sentence?” arXiv preprint arXiv:1905.07830 , 2019
Original
2019
Earlier work this paper cites.
L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima et al. , “The pile: An 800gb dataset of diverse text for language modeling,” arXiv preprint arXiv:2101.00027 , 2020
Original
2020
Earlier work this paper cites.
A. Golatkar, A. Achille, and S. Soatto, “Eternal sunshine of the spotless net: Selective forgetting in deep networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 9304–9312
2020
Earlier work this paper cites.
C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten, “Certified data removal from machine learning models,” in International Conference on Machine Learning , 2020
2020
Earlier work this paper cites.
S. Garg, S. Goldwasser, and P. N. Vasudevan, “Formalizing data deletion in the context of the right to be forgotten,” in Annual International Conference on the Theory and Applications of Cryptographic Techniques . Springer, 2020, pp. 373–402
2020
Earlier work this paper cites.
L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in 2021 IEEE Symposium on Security and Privacy (SP) . IEEE, 2021, pp. 141–159
2021
Earlier work this paper cites.
V. Gupta, C. Jung, S. Neel, A. Roth, S. Sharifi-Malvajerdi, and C. Waites, “Adaptive machine unlearning,” Advances in Neural Information Processing Systems , vol. 34, pp. 16 319–16 330, 2021
2021
Earlier work this paper cites.
Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh, “Calibrate before use: Improving few-shot performance of language models,” in International conference on machine learning . PMLR, 2021, pp. 12 697–12 706
2021
Earlier work this paper cites.
S. Neel, A. Roth, and S. Sharifi-Malvajerdi, “Descent-to-delete: Gradient-based methods for machine unlearning,” in Algorithmic Learning Theory . PMLR, 2021, pp. 931–962
2021
Earlier work this paper cites.
D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” in International Conference on Learning Representations , 2021
2021
Earlier work this paper cites.
S. Lin, J. Hilton, and O. Evans, “Truthfulqa: Measuring how models mimic human falsehoods,” arXiv preprint arXiv:2109.07958 , 2021
Original
2021
Earlier work this paper cites.
A. Sekhari, J. Acharya, G. Kamath, and A. T. Suresh, “Remember what you want to forget: Algorithms for machine unlearning,” Advances in Neural Information Processing Systems , vol. 34, pp. 18 075–18 086, 2021
2021
Earlier work this paper cites.
H. Brown, K. Lee, F. Mireshghallah, R. Shokri, and F. Tramèr, “What does it mean for a language model to preserve privacy?” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , 2022, pp. 2280–2292
2022
Earlier work this paper cites.
A. Thudi, G. Deza, V. Chandrasekaran, and N. Papernot, “Unrolling sgd: Understanding factors influencing machine unlearning,” in 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) . IEEE, 2022, pp. 303–319
2022
Earlier work this paper cites.
M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou et al. , “Challenging big-bench tasks and whether chain-of-thought can solve them,” arXiv preprint arXiv:2210.09261 , 2022
Original
2022
Earlier work this paper cites.
E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving, “Red teaming language models with language models,” arXiv preprint arXiv:2202.03286 , 2022
Original
2022
Earlier work this paper cites.
M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, and Y. Zhang, “Graph unlearning,” in Proceedings of the 2022 ACM SIGSAC conference on computer and communications security , 2022, pp. 499–513
2022
Earlier work this paper cites.
F. Tramèr, G. Kamath, and N. Carlini, “Position: Considerations for differentially private learning with large-scale public pretraining,” in Forty-first International Conference on Machine Learning , 2022
2022
Earlier work this paper cites.
N. Carlini, M. Jagielski, C. Zhang, N. Papernot, A. Terzis, and F. Tramer, “The privacy onion effect: Memorization is relative,” Advances in Neural Information Processing Systems , vol. 35, pp. 13 263–13 276, 2022
2022
Earlier work this paper cites.
J. Gao, S. Garg, M. Mahmoody, and P. N. Vasudevan, “Deletion inference, reconstruction, and compliance in machine (un) learning,” arXiv preprint arXiv:2202.03460 , 2022
Original
2022
Earlier work this paper cites.
R. Eldan and M. Russinovich, “Who’s harry potter? approximate unlearning in llms,” arXiv preprint arXiv:2310.02238 , 2023
Original
2023
Earlier work this paper cites.
P. Wang, L. Li, L. Chen, Z. Cai, D. Zhu, B. Lin, Y. Cao, Q. Liu, T. Liu, and Z. Sui, “Large language models are not fair evaluators,” arXiv preprint arXiv:2305.17926 , 2023
Original
2023
Earlier work this paper cites.
P. Pezeshkpour and E. Hruschka, “Large language models sensitivity to the order of options in multiple-choice questions,” arXiv preprint arXiv:2308.11483 , 2023
Original
2023
Earlier work this paper cites.
H. Xu, T. Zhu, L. Zhang, W. Zhou, and P. S. Yu, “Machine unlearning: A survey,” ACM Computing Surveys , vol. 56, no. 1, pp. 1–36, 2023
2023
Earlier work this paper cites.

Similar

N. Si, H. Zhang, H. Chang, W. Zhang, D. Qu, and W. Zhang, “Knowledge unlearning for llms: Tasks, methods, and challenges,” arXiv preprint arXiv:2311.15766 , 2023
Original
2023
Cited alongside, same era.
M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee, “Scalable extraction of training data from (production) language models,” arXiv preprint arXiv:2311.17035 , 2023
Original
2023
Cited alongside, same era.
V. Patil, P. Hase, and M. Bansal, “Can sensitive information be deleted from llms? objectives for defending against extraction attacks,” arXiv preprint arXiv:2309.17410 , 2023
Original
2023
Cited alongside, same era.
T.-Y. Chang, J. Thomason, and R. Jia, “Do localization methods actually localize memorized data in llms?” arXiv preprint arXiv:2311.09060 , 2023
Original
2023
Cited alongside, same era.

Then

N. Carlini, H. Cooper, F. Tramer, and C. Zhang, “Lm extraction benchmark,” https://github.com/google-research/lm-extraction-benchmark , 2023, accessed: 2024-09-24
2024
Closest in time.
J. Ji, D. Hong, B. Zhang, B. Chen, J. Dai, B. Zheng, T. Qiu, B. Li, and Y. Yang, “Pku-saferlhf: A safety alignment preference dataset for llama family models,” arXiv preprint arXiv:2406.15513 , 2024
Original
2024
Closest in time.
L. Wang, X. Zeng, J. Guo, K.-F. Wong, and G. Gottlob, “Selective forgetting: Advancing machine unlearning techniques and evaluation in language models,” arXiv preprint arXiv:2402.05813 , 2024
Original
2024
Closest in time.
G.-O. Barbulescu and P. Triantafillou, “To each (textual sequence) its own: Improving memorized-data unlearning in large language models,” arXiv preprint arXiv:2405.03097 , 2024
Original
2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

P. Hase, M. Bansal, B. Kim, and A. Ghandeharioun, “Does localization inform editing? surprising differences in causality-based localization vs,” Knowledge Editing in Language Models , 2023

2023

Cited alongside, same era.

H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine et al. , “Llama guard: Llm-based input-output safeguard for human-ai conversations,” arXiv preprint arXiv:2312.06674 , 2023

2023

Cited alongside, same era.

T. Rebedea, R. Dinu, M. Sreedhar, C. Parisien, and J. Cohen, “Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails,” arXiv preprint arXiv:2310.10501 , 2023

2023

Cited alongside, same era.

W. Shi, A. Ajith, M. Xia, Y. Huang, D. Liu, T. Blevins, D. Chen, and L. Zettlemoyer, “Detecting pretraining data from large language models,” arXiv preprint arXiv:2310.16789 , 2023

2023

Cited alongside, same era.

A. Cohen, A. Smith, M. Swanberg, and P. N. Vasudevan, “Control, confidentiality, and the right to be forgotten,” in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security , 2023, pp. 3358–3372

2023

Cited alongside, same era.

Y. Huang and C. L. Canonne, “Tight bounds for machine unlearning via differential privacy,” arXiv preprint arXiv:2309.00886 , 2023

2023

Cited alongside, same era.

A. Muresanu, A. Thudi, M. R. Zhang, and N. Papernot, “Unlearnable algorithms for in-context learning,” arXiv preprint arXiv:2402.00751 , 2024

2024

Cited alongside, same era.

N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, J. D. Li, A.-K. Dombrowski, S. Goel, L. Phan et al. , “The WMDP benchmark: Measuring and reducing malicious use with unlearning,” in International Conference on Machine Learning , 2024

2024

Cited alongside, same era.

S. Cha, S. Cho, D. Hwang, and M. Lee, “Towards robust and cost-efficient knowledge unlearning for large language models,” arXiv preprint arXiv:2408.06621 , 2024

2024

Closest in time.

D. Huu-Tien, T.-T. Pham, H. Thanh-Tung, and N. Inoue, “On effects of steering latent representation for large language model unlearning,” arXiv preprint arXiv:2408.06223 , 2024

2024

Closest in time.

R. Tamirisa, B. Bharathi, L. Phan, A. Zhou, A. Gatti, T. Suresh, M. Lin, J. Wang, R. Wang, R. Arel et al. , “Tamper-resistant safeguards for open-weight llms,” arXiv preprint arXiv:2408.00761 , 2024

2024

Closest in time.

A. Kolbeinsson, K. O’Brien, T. Huang, S. Gao, S. Liu, J. R. Schwarz, A. Vaidya, F. Mahmood, M. Zitnik, T. Chen et al. , “Composable interventions for language models,” arXiv preprint arXiv:2407.06483 , 2024

2024

Closest in time.

N. Li, Z. Han, I. Steneker, W. Primack, R. Goodside, H. Zhang, Z. Wang, C. Menghini, and S. Yue, “Llm defenses are not robust to multi-turn human jailbreaks yet,” arXiv preprint arXiv:2408.15221 , 2024

2024

Closest in time.

C. Y. Liu, Y. Wang, J. Flanigan, and Y. Liu, “Large language model unlearning via embedding-corrupted prompts,” arXiv preprint arXiv:2406.07933 version 1 , 12 Jun 2024

2024

Closest in time.

Q. Wang, B. Han, P. Yang, J. Zhu, T. Liu, and M. Sugiyama, “Unlearning with control: Assessing real-world utility for large language model unlearning,” arXiv preprint arXiv:2406.09179 , 2024

2024

Closest in time.

H. Yuan, Z. Jin, P. Cao, Y. Chen, K. Liu, and J. Zhao, “Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models,” arXiv preprint arXiv:2408.10682 , 2024

2024

Closest in time.

A. Lynch, P. Guo, A. Ewart, S. Casper, and D. Hadfield-Menell, “Eight methods to evaluate robust unlearning in llms,” arXiv preprint arXiv:2402.16835 , 2024

2024

Closest in time.

W. Shi, J. Lee, Y. Huang, S. Malladi, J. Zhao, A. Holtzman, D. Liu, L. Zettlemoyer, N. A. Smith, and C. Zhang, “Muse: Machine unlearning six-way evaluation for language models,” arXiv preprint arXiv:2407.06460 , 2024

2024

Closest in time.

Y. Hong, L. Yu, S. Ravfogel, H. Yang, and M. Geva, “Intrinsic evaluation of unlearning using parametric knowledge traces,” arXiv preprint arXiv:2406.11614 , 2024

2024

Closest in time.

S. Hu, Y. Fu, Z. S. Wu, and V. Smith, “Jogging the memory of unlearned model through targeted relearning attack,” arXiv preprint arXiv:2406.13356 , 2024

2024

Closest in time.

R. Tamirisa, B. Bharathi, A. Zhou, and B. L. M. Mazeika, “Toward robust unlearning for llms,” in ICLR 2024 Workshop on Secure and Trustworthy Large Language Models , 2024

2024

Closest in time.

H. Gao, T. Pang, C. Du, T. Hu, Z. Deng, and M. Lin, “Meta-unlearning on diffusion models: Preventing relearning unlearned concepts,” arXiv preprint arXiv:2410.12777 , 2024

Original

2024

Closest in time.

N. Mangaokar, A. Hooda, J. Choi, S. Chandrashekaran, K. Fawaz, S. Jha, and A. Prakash, “Prp: Propagating universal perturbations to attack large language model guard-rails,” arXiv preprint arXiv:2402.15911 , 2024

2024

Closest in time.

R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.

A. Sheshadri, A. Ewart, P. Guo, A. Lynch, C. Wu, V. Hebbar, H. Sleight, A. C. Stickland, E. Perez, D. Hadfield-Menell et al. , “Targeted latent adversarial training improves robustness to persistent harmful behaviors in llms,” arXiv preprint arXiv:2407.15549 , 2024

2024

Closest in time.

T. Kim, S. Kotha, and A. Raghunathan, “Jailbreaking is best solved by definition,” arXiv preprint arXiv:2403.14725 , 2024

2024

Closest in time.

P. Thaker, Y. Maurya, and V. Smith, “Guardrail baselines for unlearning in llms,” arXiv preprint arXiv:2403.03329 , 2024

2024

Closest in time.

G. Chen, Y. Wang, H. Sun, and G. Chen, “Wpn: An unlearning method based on n-pair contrastive learning in language models,” arXiv preprint arXiv:2408.09459 , 2024

2024

Closest in time.

K. Chen, Z. Wang, B. Mi, W. Liu, S. Wang, X. Ren, and J. Shen, “Machine unlearning in large language models,” arXiv preprint arXiv:2404.16841 , 2024

2024

Closest in time.

J. Zhang, J. Sun, E. Yeats, Y. Ouyang, M. Kuo, J. Zhang, H. Yang, and H. Li, “Min-k%++: Improved baseline for detecting pre-training data from large language models,” arXiv preprint arXiv:2404.02936 , 2024

2024

Closest in time.

R. Zhang, M. Bertran, and A. Roth, “Order of magnitude speedups for llm membership inference,” arXiv preprint arXiv:2409.14513 , September 2024

2024

Closest in time.

S. B. R. Chowdhury, K. Choromanski, A. Sehanobish, A. Dubey, and S. Chaturvedi, “Towards scalable exact machine unlearning using parameter-efficient fine-tuning,” arXiv preprint arXiv:2406.16257 , 2024

2024

Closest in time.

D. Jeon, W. Jeung, T. Kim, A. No, and J. Choi, “An information theoretic metric for evaluating unlearning models,” arXiv preprint arXiv:2405.17878 , 2024

2024

Closest in time.