RKLD: Reverse KL-Divergence-based Knowledge Distillation for Unlearning Personal Information in Large Language Models

Distilling the knowledge in a neural network

Original

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015 · 2015

Earlier work this paper cites.

f-divergence inequalities via functional domination

Igal Sason and Sergio Verdú. 2016 · 2016

Earlier work this paper cites.

The eu general data protection regulation (gdpr)

Paul Voigt and Axel Von dem Bussche. 2017 · 2017

Earlier work this paper cites.

Eternal sunshine of the spotless net: Selective forgetting in deep networks

Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020 · 2020

Earlier work this paper cites.

Dataset inference: Ownership resolution in machine learning

Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot. 2020 · 2020

Earlier work this paper cites.

Machine unlearning

Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021 · 2021

Earlier work this paper cites.

Extracting training data from large language models

Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021 · 2021

Earlier work this paper cites.

Remember what you want to forget: Algorithms for machine unlearning

Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. 2021 · 2021

Earlier work this paper cites.

Quantifying memorization across neural language models

Original

Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022 · 2022

Earlier work this paper cites.

Efficient model updates for approximate unlearning of graph-structured data

Eli Chien, Chao Pan, and Olgica Milenkovic. 2022 · 2022

Earlier work this paper cites.

Federated unlearning: How to efficiently erase a client in fl?

Original

Anisa Halimi, Swanand Kadhe, Ambrish Rawat, and Nathalie Baracaldo. 2022 · 2022

Earlier work this paper cites.

Editing models with task arithmetic

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2022 · 2022

Earlier work this paper cites.

Knowledge unlearning for mitigating privacy risks in language models

Original

Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2022 · 2022

Cited alongside, same era.

uChecker: Masked pretrained language models as unsupervised Chinese spelling checkers

Piji Li. 2022 · 2022

Cited alongside, same era.

Quark: Controllable text generation with reinforced unlearning

Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022 · 2022

Cited alongside, same era.

PhysNLU: A language resource for evaluating natural language understanding and explanation coherence in physics

Jordan Meadows, Zili Zhou, and André Freitas. 2022 · 2022

Cited alongside, same era.

Unlearn what you want to forget: Efficient unlearning for llms

Jiaao Chen and Diyi Yang. 2023 · 2023

Cited alongside, same era.

Kga: A general machine unlearning framework based on knowledge gap alignment

Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. 2023 · 2023

Later among the works it cites.

Certified edge unlearning for graph neural networks

Kun Wu, Jie Shen, Yue Ning, Ting Wang, and Wendy Hui Wang. 2023 · 2023

Later among the works it cites.

Large language model unlearning

Original

Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2023 · 2023

Later among the works it cites.

Gradient ascent post-training enhances language model generalization

Dongkeun Yoon, Joel Jang, Sungdong Kim, and Minjoon Seo. 2023 · 2023

Later among the works it cites.

Unlearning bias in language models by partitioning gradients

Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. 2023 · 2023

Later among the works it cites.