Fetching the paper…
Reading the bibliography…
We train language models (LMs) with federated learning (FL) and differential privacy (DP) in the Google Keyboard (Gboard).
The algorithmic foundations of differential privacy
Cynthia Dwork, Aaron Roth, et al · 2014
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
Concentrated differential privacy: Simplifications, extensions, and lower bounds
Mark Bun and Thomas Steinke · 2016
Earlier work this paper cites.
Practical secure aggregation for privacy-preserving machine learning
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth · 2017
Earlier work this paper cites.
Differentially private federated learning: A client level perspective
Robin C Geyer, Tassilo Klein, and Moin Nabi · 2017
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas · 2017
Earlier work this paper cites.
Compressing word embeddings via deep compositional code learning
Raphael Shu and Hideki Nakayama · 2017
Earlier work this paper cites.
Federated learning for mobile keyboard prediction
Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage · 2018
Earlier work this paper cites.
Learning differentially private recurrent language models
Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang · 2018
Earlier work this paper cites.
Applied federated learning: Improving google keyboard query suggestions
Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Françoise Beaufays · 2018
Earlier work this paper cites.
Towards federated learning at scale: System design
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečnỳ, Stefano Mazzocchi, Brendan McMahan, et al · 2019
Earlier work this paper cites.
The secret sharer: Evaluating and testing unintended memorization in neural networks
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song · 2019
Earlier work this paper cites.
Advances and open problems in federated learning
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kaylee Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D’Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaïd Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konecný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao · 2019
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 2019
Cited alongside, same era.
Privacy amplification via random check-ins
Borja Balle, Peter Kairouz, Brendan McMahan, Om Thakkar, and Abhradeep Guha Thakurta · 2020
Cited alongside, same era.
Secure single-server aggregation with (poly) logarithmic overhead
James Henry Bell, Kallista A Bonawitz, Adrià Gascón, Tancrède Lepoint, and Mariana Raykova · 2020
Differentially private fine-tuning of language models
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al · 2021
Later among the works it cites.
The fundamental price of secure aggregation in differentially private federated learning
Wei-Ning Chen, Christopher A Choquette Choo, Peter Kairouz, and Ananda Theertha Suresh · 2022
Later among the works it cites.
Multi-epoch matrix factorization mechanisms for private machine learning
Christopher A Choquette-Choo, H Brendan McMahan, Keith Rush, and Abhradeep Thakurta · 2022
Later among the works it cites.
Unlocking high-accuracy differentially private image classification through scale
Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle · 2022
Later among the works it cites.
Google’s differential privacy libraries., 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Training production language models without memorizing user data
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, Galen Andrew, H Brendan McMahan, and Françoise Beaufays · 2020
Cited alongside, same era.
mt5: A massively multilingual pre-trained text-to-text transformer
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel · 2020
Cited alongside, same era.
The skellam mechanism for differentially private federated learning
Naman Agarwal, Peter Kairouz, and Ziyu Liu · 2021
Cited alongside, same era.
Differentially private learning with adaptive clipping
Galen Andrew, Om Thakkar, H Brendan McMahan, and Swaroop Ramaswamy · 2021
Cited alongside, same era.
Federated learning and privacy: Building privacy-preserving systems for machine learning and data science on decentralized data
Kallista Bonawitz, Peter Kairouz, Brendan McMahan, and Daniel Ramage · 2021
Cited alongside, same era.
Large language models can be strong differentially private learners
Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto · 2021
Cited alongside, same era.
Disclosure avoidance for the 2020 census: An introduction, 2021
US Census Bureau · 2021
Cited alongside, same era.
A field guide to federated optimization
Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H Brendan McMahan, Blaise Aguera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, et al · 2021
Cited alongside, same era.
DP Team · 2022
Later among the works it cites.
Federated learning with formal differential privacy guarantees, 2022
Brendan McMahan and Abhradeep Thakurta · 2022
Later among the works it cites.
TensorFlow Privacy., 2022
TFP Authors · 2022
Later among the works it cites.
Learning to generate image embeddings with user-level differential privacy
Zheng Xu, Maxwell Collins, Yuxiao Wang, Liviu Panait, Sewoong Oh, Sean Augenstein, Ting Liu, Florian Schroff, and H Brendan McMahan · 2022
Later among the works it cites.
On the convergence of federated averaging with cyclic client participation
Yae Jee Cho, Pranay Sharma, Gauri Joshi, Zheng Xu, Satyen Kale, and Tong Zhang · 2023
Closest in time.
How to dp-fy ml: A practical guide to machine learning with differential privacy, 2023
Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H. Brendan McMahan, Sergei Vassilvitskii, Steve Chien, and Abhradeep Thakurta · 2023
Closest in time.
Can public large language models help private cross-device federated learning?
Boxin Wang, Yibo Jacky Zhang, Yuan Cao, Bo Li, H Brendan McMahan, Sewoong Oh, Zheng Xu, and Manzil Zaheer · 2023
Closest in time.
Diurnal or nocturnal? federated learning of multi-branch networks from periodically shifting distributions
Chen Zhu, Zheng Xu, Mingqing Chen, Jakub Konečnỳ, Andrew Hard, and Tom Goldstein · 2023
Closest in time.