Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning

J. Rosen, “The right to be forgotten,” Stan. L. Rev. Online , vol. 64, p. 88, 2011

2011

Earlier work this paper cites.

Y. Cao and J. Yang, “Towards making systems forget with machine unlearning,” in 2015 IEEE symposium on security and privacy . IEEE, 2015, pp. 463–480

2015

Earlier work this paper cites.

P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems , vol. 30, 2017

2017

Earlier work this paper cites.

I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101 , 2017

Original

2017

Earlier work this paper cites.

C. J. Hoofnagle, B. van der Sloot, and F. Z. Borgesius, “The european union general data protection regulation: what it is and what it means,” Information & Communications Technology Law , vol. 28, no. 1, pp. 65–98, 2019

2019

Earlier work this paper cites.

D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irving, “Fine-tuning language models from human preferences,” arXiv preprint arXiv:1909.08593 , 2019

Original

2019

Earlier work this paper cites.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

2019

Earlier work this paper cites.

A. Warnecke, L. Pirch, C. Wressnegger, and K. Rieck, “Machine unlearning of features and labels,” arXiv preprint arXiv:2108.11577 , 2021

Original

2021

Earlier work this paper cites.

L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in 2021 IEEE Symposium on Security and Privacy (SP) . IEEE, 2021, pp. 141–159

2021

Earlier work this paper cites.

A. Thudi, G. Deza, V. Chandrasekaran, and N. Papernot, “Unrolling sgd: Understanding factors influencing machine unlearning,” in 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) . IEEE, 2022, pp. 303–319

2022

Earlier work this paper cites.

B. Liu, Q. Liu, and P. Stone, “Continual learning and private unlearning,” in Conference on Lifelong Learning Agents . PMLR, 2022, pp. 243–254

2022

Earlier work this paper cites.

T. T. Nguyen, T. T. Huynh, P. L. Nguyen, A. W.-C. Liew, H. Yin, and Q. V. H. Nguyen, “A survey of machine unlearning,” arXiv preprint arXiv:2209.02299 , 2022

Original

2022

Earlier work this paper cites.

Y. Liu, M. Fan, C. Chen, X. Liu, Z. Ma, L. Wang, and J. Ma, “Backdoor defense with machine unlearning,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications . IEEE, 2022, pp. 280–289

2022

Earlier work this paper cites.

Y. Liu, L. Xu, X. Yuan, C. Wang, and B. Li, “The right to be forgotten in federated learning: An efficient realization with rapid retraining,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications . IEEE, 2022, pp. 1749–1758

2022

Earlier work this paper cites.

A. Halimi, S. Kadhe, A. Rawat, and N. Baracaldo, “Federated unlearning: How to efficiently erase a client in fl?” arXiv preprint arXiv:2207.05521 , 2022

Original

2022

Earlier work this paper cites.

M. Chen, Z. Zhang, T. Wang, M. Backes, M. Humbert, and Y. Zhang, “Graph unlearning,” in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security , 2022, pp. 499–513

2022

Earlier work this paper cites.

E. Chien, C. Pan, and O. Milenkovic, “Certified graph unlearning,” arXiv preprint arXiv:2206.09140 , 2022

Original

2022

Earlier work this paper cites.

X. Lu, S. Welleck, J. Hessel, L. Jiang, L. Qin, P. West, P. Ammanabrolu, and Y. Choi, “Quark: Controllable text generation with reinforced unlearning,” Advances in neural information processing systems , vol. 35, pp. 27 591–27 609, 2022

2022

Earlier work this paper cites.

J. Jang, D. Yoon, S. Yang, S. Cha, M. Lee, L. Logeswaran, and M. Seo, “Knowledge unlearning for mitigating privacy risks in language models,” arXiv preprint arXiv:2210.01504 , 2022

Original

2022

Earlier work this paper cites.

V. B. Kumar, R. Gangadharaiah, and D. Roth, “Privacy adhering machine un-learning in nlp,” arXiv preprint arXiv:2212.09573 , 2022

Original

2022

Earlier work this paper cites.

G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi, “Editing models with task arithmetic,” arXiv preprint arXiv:2212.04089 , 2022

Original

2022

Earlier work this paper cites.

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al. , “Training language models to follow instructions with human feedback,” Advances in neural information processing systems , vol. 35, pp. 27 730–27 744, 2022

2022

Earlier work this paper cites.

B. Wang, W. Chen, H. Pei, C. Xie, M. Kang, C. Zhang, C. Xu, Z. Xiong, R. Dutta, R. Schaeffer et al. , “Decodingtrust: A comprehensive assessment of trustworthiness in gpt models.” in NeurIPS , 2023

2023

Earlier work this paper cites.

J. Jia, J. Liu, P. Ram, Y. Yao, G. Liu, Y. Liu, P. Sharma, and S. Liu, “Model sparsity can simplify machine unlearning,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023

2023

Cited alongside, same era.

R. Gandikota, J. Materzynska, J. Fiotto-Kaufman, and D. Bau, “Erasing concepts from diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 2426–2436

2023

Cited alongside, same era.

R. Eldan and M. Russinovich, “Who’s harry potter? approximate unlearning in llms,” 2023

2023

Cited alongside, same era.

Y. Yao, X. Xu, and Y. Liu, “Large language model unlearning,” arXiv preprint arXiv:2310.10683 , 2023

Original

2023

Cited alongside, same era.

R. Jin, M. Chen, Q. Zhang, and X. Li, “Forgettable federated linear learning with certified data removal,” arXiv preprint arXiv:2306.02216 , 2023

J. Jia, Y. Zhang, Y. Zhang, J. Liu, B. Runwal, J. Diffenderfer, B. Kailkhura, and S. Liu, “Soul: Unlocking the power of second-order optimization for llm unlearning,” arXiv preprint arXiv:2404.18239 , 2024

Original

2024

Closest in time.

R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.

Y. Meng, M. Xia, and D. Chen, “SimPO: Simple preference optimization with a reference-free reward,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024

2024

Closest in time.

A. Lynch, P. Guo, A. Ewart, S. Casper, and D. Hadfield-Menell, “Eight methods to evaluate robust unlearning in llms,” arXiv preprint arXiv:2402.16835 , 2024

Original

2024

Closest in time.