2020

Learn to Forget: Machine Unlearning via Neuron Masking

Liu, Yang, Ma, Zhuo, Liu, Ximeng et al.

Understand

Nowadays, machine learning models, especially neural networks, become prevalent in many real-world applications.These models are trained based on a one-way trip from user data: as long as users contribute their data, there is no way to withdraw; and it is well-known that a neural network memorizes its training data.

This contradicts the "right to be forgotten" clause of GDPR, potentially leading to law violations.
To this end, machine unlearning becomes a popular research topic, which allows users to eliminate memorization of their private data from a trained machine learning model.In this paper, we propose the first uniform metric called for-getting rate to measure the effectiveness of a machine unlearning method.
It is based on the concept of membership inference and describes the transformation rate of the eliminated data from "memorized" to "unknown" after conducting unlearning.

Built on

A. O’hagan and T. Leonard, “Bayes estimation subject to uncertainty about parameter constraints,” Biometrika , vol. 63, no. 1, pp. 201–203, 1976
1976
Earlier work this paper cites.
J. Lin, “Divergence measures based on the shannon entropy,” IEEE Transactions on Information theory , vol. 37, no. 1, pp. 145–151, 1991
1991
Earlier work this paper cites.
M. Marcus, B. Santorini, and M. A. Marcinkiewicz, “Building a large annotated corpus of english: The penn treebank,” 1993
1993
Earlier work this paper cites.
J. R. Quinlan et al. , “Bagging, boosting, and c4. 5,” in AAAI/IAAI, Vol. 1 , 1996, pp. 725–730
1996
Earlier work this paper cites.
C. Zhu, R. H. Byrd, P. Lu, and J. Nocedal, “Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization,” ACM Transactions on Mathematical Software (TOMS) , vol. 23, no. 4, pp. 550–560, 1997
1997
Earlier work this paper cites.
M. Kearns, “Efficient noise-tolerant learning from statistical queries,” Journal of the ACM (JACM) , vol. 45, no. 6, pp. 983–1006, 1998
1998
Earlier work this paper cites.
C. E. Shannon, “A mathematical theory of communication,” ACM SIGMOBILE mobile computing and communications review , vol. 5, no. 1, pp. 3–55, 2001
2001
Earlier work this paper cites.
D. L. Donoho, “For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution,” Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences , vol. 59, no. 6, pp. 797–829, 2006
2006
Earlier work this paper cites.
L. v. d. Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of machine learning research , vol. 9, no. Nov, pp. 2579–2605, 2008
2008
Earlier work this paper cites.
M. Sundermeyer, R. Schlüter, and H. Ney, “Lstm neural networks for language modeling,” in 13t Annual Conference of the International Speech Communication Association , 2012
2012
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Earlier work this paper cites.
Y. Cao and J. Yang, “Towards making systems forget with machine unlearning,” in 2015 IEEE Symposium on Security and Privacy (S&P) . IEEE, 2015, pp. 463–480
2015
Earlier work this paper cites.
I. Goodfellow, “Nips 2016 tutorial: Generative adversarial networks,” Advances in Neural Information Processing Systems (NIPS) , 2016
2016
Earlier work this paper cites.
C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning requires rethinking generalization,” arXiv preprint arXiv:1611.03530 , 2016
Original
2016
Earlier work this paper cites.
T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2016, pp. 785–794
2016
Earlier work this paper cites.
F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction apis,” in 25th { \{ USENIX } \} Security Symposium ( { \{ USENIX } \} Security 16) , 2016, pp. 601–618
2016
Earlier work this paper cites.
K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . ACM, 2017, pp. 1175–1191
2017
Earlier work this paper cites.
P. Voigt and A. Von dem Bussche, “The eu general data protection regulation (gdpr),” A Practical Guide, 1st Ed., Cham: Springer International Publishing , 2017
2017
Earlier work this paper cites.

Similar

R. C. Geyer, T. Klein, and M. Nabi, “Differentially private federated learning: A client level perspective,” arXiv preprint arXiv:1712.07557 , 2017
Original
2017
Cited alongside, same era.
R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE Symposium on Security and Privacy (S&P) . IEEE, 2017, pp. 3–18
2017
Cited alongside, same era.
R. L. Davis and Y. Zhong, “The biology of forgetting—a perspective,” Neuron , vol. 95, no. 3, pp. 490–503, 2017
2017
Cited alongside, same era.
J. Liu, M. Juuti, Y. Lu, and N. Asokan, “Oblivious neural network predictions via minionn transformations,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security , 2017, pp. 619–631
2017
Cited alongside, same era.
V. Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Federated multi-task learning,” in Advances in Neural Information Processing Systems (NIPS , 2017, pp. 4424–4434
2017
Cited alongside, same era.
P. Mohassel and P. Rindal, “Aby3: A mixed protocol framework for machine learning,” in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security , 2018, pp. 35–52
2018
Cited alongside, same era.
M. A. Rahman, T. Rahman, R. Laganière, N. Mohammed, and Y. Wang, “Membership inference attack against differentially private deep learning model.” Trans. Data Priv. , vol. 11, no. 1, pp. 61–79, 2018
2018
Cited alongside, same era.
M. Nasr, R. Shokri, and A. Houmansadr, “Machine learning with membership privacy using adversarial regularization,” in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security , 2018, pp. 634–646
2018
Cited alongside, same era.
D. Hendrycks, M. Mazeika, and T. Dietterich, “Deep anomaly detection with outlier exposure,” in International Conference on Learning Representations (ICLR) , 2018
2018
Cited alongside, same era.
S. Truex, L. Liu, M. E. Gursoy, L. Yu, and W. Wei, “Towards demystifying membership inference attacks,” arXiv preprint arXiv:1807.09173 , 2018
Original
2018
Cited alongside, same era.
N. H. Tran, W. Bao, A. Zomaya, N. M. NH, and C. S. Hong, “Federated learning over wireless networks: Optimization model design and analysis,” in 2019 IEEE Conference on Computer Communications (INFOCOM) . IEEE, 2019, pp. 1387–1395
2019
Cited alongside, same era.
N. Agrawal, A. Shahin Shamsabadi, M. J. Kusner, and A. Gascón, “Quotient: two-party secure neural network training and prediction,” in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security , 2019, pp. 1231–1247
2019
Cited alongside, same era.

Then

S. Bock and M. Weiß, “A proof of local convergence for the adam optimizer,” in 2019 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2019, pp. 1–8
2019
Later among the works it cites.
Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST) , vol. 10, no. 2, pp. 1–19, 2019
2019
Later among the works it cites.
M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,” in 2019 IEEE symposium on security and privacy (SP) . IEEE, 2019, pp. 739–753
2019
Later among the works it cites.
J. Jia, A. Salem, M. Backes, Y. Zhang, and N. Z. Gong, “Memguard: Defending against black-box membership inference attacks via adversarial examples,” in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security , 2019, pp. 259–274
2019
Later among the works it cites.
C. Guo, T. Goldstein, A. Hannun, and L. van der Maaten, “Certified data removal from machine learning models,” arXiv preprint arXiv:1911.03030 , 2019
Original
2019
Later among the works it cites.
R. Xu, N. Baracaldo, Y. Zhou, A. Anwar, and H. Ludwig, “Hybridalpha: An efficient approach for privacy-preserving federated learning,” in Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security , 2019, pp. 13–23
2019
Later among the works it cites.
V. Feldman, “Does learning require memorization? a short tale about a long tail,” in Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing , 2020, pp. 954–959
2020
Closest in time.
A. Salem, A. Bhattacharya, M. Backes, M. Fritz, and Y. Zhang, “Updates-leak: Data set inference and reconstruction attacks in online learning,” in 29th { \{ USENIX } \} Security Symposium ( { \{ USENIX } \} Security 20) , 2020, pp. 1291–1308
2020
Closest in time.
Y.-C. Hsu, Y. Shen, H. Jin, and Z. Kira, “Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 10 951–10 960
2020
Closest in time.
G. Liu, Y. Yang, X. Ma, C. Wang, and J. Liu, “Federated unlearning,” arXiv preprint arXiv:2012.13891 , 2020
Original
2020
Closest in time.
K. Leino and M. Fredrikson, “Stolen memories: Leveraging model memorization for calibrated white-box membership inference,” in 29th { \{ USENIX } \} Security Symposium ( { \{ USENIX } \} Security 20) , 2020, pp. 1605–1622
2020
Closest in time.
A. B. Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. García, S. Gil-López, D. Molina, R. Benjamins et al. , “Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai,” Information Fusion , vol. 58, pp. 82–115, 2020
2020
Closest in time.
V. Tolpegin, S. Truex, M. E. Gursoy, and L. Liu, “Data poisoning attacks against federated learning systems,” in European Symposium on Research in Computer Security . Springer, 2020, pp. 480–501
2020
Closest in time.
V. Feldman and C. Zhang, “What neural networks memorize and why: Discovering the long tail via influence estimation,” arXiv preprint arXiv:2008.03703 , 2020
Original
2020
Closest in time.
Z. Jiang, C. Zhang, K. Talwar, and M. C. Mozer, “Characterizing structural regularities of labeled data in overparameterized models,” arXiv e-prints , pp. arXiv–2002, 2020
2020
Closest in time.
L. Bourtoule, V. Chandrasekaran, C. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” 2021 IEEE Symposium on Security and Privacy (S&P) , 2021
2021
Closest in time.
S. Neel, A. Roth, and S. Sharifi-Malvajerdi, “Descent-to-delete: Gradient-based methods for machine unlearning,” in Algorithmic Learning Theory . PMLR, 2021, pp. 931–962
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…