2020

Training Production Language Models without Memorizing User Data

Ramaswamy, Swaroop, Thakkar, Om, Mathews, Rajiv et al.

Understand

This paper presents the first consumer-scale next-word prediction (NWP) model trained with Federated Learning (FL) while leveraging the Differentially Private Federated Averaging (DP-FedAvg) technique.

There has been prior work on building practical FL infrastructure, including work demonstrating the feasibility of training language models on mobile devices using such infrastructure.
It has also been shown (in simulations on a public corpus) that it is possible to train NWP models with user-level differential privacy using the DP-FedAvg algorithm.
Nevertheless, training production-quality NWP models with DP-FedAvg in a real-world production environment on a heterogeneous fleet of mobile phones requires addressing numerous challenges.

Built on

Our data, ourselves: Privacy via distributed noise generation
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor · 2006
Earlier work this paper cites.
Our data, ourselves: Privacy via distributed noise generation
Cynthia Dwork, Krishnaram Kenthapadi, Frank Mcsherry, Ilya Mironov, and Moni Naor · 2006
Earlier work this paper cites.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith · 2006
Earlier work this paper cites.
Differentially private combinatorial optimization
Anupam Gupta, Katrina Ligett, Frank McSherry, Aaron Roth, and Kunal Talwar · 2010
Earlier work this paper cites.
Differentially private empirical risk minimization
Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate · 2011
Earlier work this paper cites.
A stability-based validation procedure for differentially private machine learning
Kamalika Chaudhuri and Staal Vinterbo · 2013
Earlier work this paper cites.
Private empirical risk minimization, revisited
Original
Raef Bassily, Adam D. Smith, and Abhradeep Thakurta · 2014
Earlier work this paper cites.
The algorithmic foundations of differential privacy
Cynthia Dwork, Aaron Roth, et al · 2014
Earlier work this paper cites.
Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition
Original
Hasim Sak, Andrew W. Senior, and Françoise Beaufays · 2014
Earlier work this paper cites.
TensorFlow: Large-scale machine learning on heterogeneous systems, 2015
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng · 2015
Earlier work this paper cites.
Model inversion attacks that exploit confidence information and basic countermeasures
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.

Similar

Semi-supervised knowledge transfer for deep learning from private training data
Original
Nicolas Papernot, Martín Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar · 2016
Cited alongside, same era.
A methodology for formalizing model-inversion attacks
X. Wu, M. Fredrikson, S. Jha, and J. F. Naughton · 2016
Cited alongside, same era.
Rényi differential privacy
I. Mironov · 2017
Cited alongside, same era.
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas · 2017
Cited alongside, same era.
Learning differentially private language models without losing accuracy
Original
H. Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang · 2017
Cited alongside, same era.
Membership inference attacks against machine learning models
R. Shokri, M. Stronati, C. Song, and V. Shmatikov · 2017
Cited alongside, same era.
Bolt-on differential privacy for scalable stochastic gradient descent-based analytics
Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, and Jeffrey Naughton · 2017
Cited alongside, same era.
The secret sharer: Measuring unintended neural network memorization & extracting secrets
Original
Nicholas Carlini, Chang Liu, Jernej Kos, Úlfar Erlingsson, and Dawn Song · 2018
Cited alongside, same era.
Federated learning for mobile keyboard prediction
Original
Andrew Hard, Kanishka Rao, Rajiv Mathews, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage · 2018
Cited alongside, same era.
Differentially private matrix completion revisited
Prateek Jain, Om Thakkar, and Abhradeep Thakurta · 2018
Cited alongside, same era.
Scalable private learning with pate
Original
Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson · 2018
Cited alongside, same era.
Element level differential privacy: The right granularity of privacy, 2019
Hilal Asi, John Duchi, and Omid Javidbakht · 2019
Cited alongside, same era.

Then

Generative models for effective ML on private, decentralized datasets
Original
Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, and Blaise Agüera y Arcas · 2019
Later among the works it cites.
Towards federated learning at scale: System design
Original
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konecný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander · 2019
Later among the works it cites.
Towards practical differentially private convex optimization
Roger Iyengar, Joseph P Near, Dawn Song, Om Thakkar, Abhradeep Thakurta, and Lun Wang · 2019
Later among the works it cites.
Private selection from private candidates
Jingcheng Liu and Kunal Talwar · 2019
Later among the works it cites.
Auditing data provenance in text-generation models
Congzheng Song and Vitaly Shmatikov · 2019
Later among the works it cites.
Differentially private learning with adaptive clipping
Original
Om Thakkar, Galen Andrew, and H. Brendan McMahan · 2019
Later among the works it cites.
Subsampled renyi differential privacy and analytical moments accountant
Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan · 2019
Later among the works it cites.
Privacy amplification via random check-ins
Original
Borja Balle, Peter Kairouz, H. Brendan McMahan, Om Thakkar, and Abhradeep Thakurta · 2020
Closest in time.
Understanding gradient clipping in private SGD: A geometric perspective
Original
Xiangyi Chen, Zhiwei Steven Wu, and Mingyi Hong · 2020
Closest in time.
Characterizing private clipped gradient descent on convex generalized linear problems
Original
Shuang Song, Om Thakkar, and Abhradeep Thakurta · 2020
Closest in time.
Understanding unintended memorization in federated learning
Original
Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews, and Françoise Beaufays · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…