Fetching the paper…
Reading the bibliography…
By ensuring differential privacy in the learning algorithms, one can rigorously mitigate the risk of large models memorizing sensitive training data.
Our data, ourselves: Privacy via distributed noise generation
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor · 2006
Earlier work this paper cites.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith · 2006
Earlier work this paper cites.
Differentially private empirical risk minimization
Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate · 2011
Earlier work this paper cites.
Adaptive subgradient methods for online learning and stochastic optimization
John Duchi, Elad Hazan, and Yoram Singer · 2011
Earlier work this paper cites.
Neural networks for machine learning lecture 6a overview of mini-batch gradient descent
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky · 2012
Earlier work this paper cites.
Stochastic first-and zeroth-order methods for nonconvex stochastic programming
Saeed Ghadimi and Guanghui Lan · 2013
Earlier work this paper cites.
Differentially private empirical risk minimization: Efficient algorithms and tight error bounds
Raef Bassily, Adam Smith, and Abhradeep Thakurta · 2014
Earlier work this paper cites.
The algorithmic foundations of differential privacy
Cynthia Dwork, Aaron Roth, et al · 2014
Earlier work this paper cites.
Analyze gauss: optimal bounds for privacy-preserving principal component analysis
Cynthia Dwork, Kunal Talwar, Abhradeep Thakurta, and Li Zhang · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
Concentrated differential privacy: Simplifications, extensions, and lower bounds
Mark Bun and Thomas Steinke · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Rényi differential privacy
Ilya Mironov · 2017
Earlier work this paper cites.
Differentially private empirical risk minimization revisited: Faster and more general
Di Wang, Minwei Ye, and Jinhui Xu · 2017
Earlier work this paper cites.
Composable and versatile privacy via truncated cdp
Mark Bun, Cynthia Dwork, Guy N Rothblum, and Thomas Steinke · 2018
Earlier work this paper cites.
Glue: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman · 2018
Earlier work this paper cites.
Group normalization
Yuxin Wu and Kaiming He · 2018
Cited alongside, same era.
Differentially private learning with adaptive clipping
Galen Andrew, Om Thakkar, H Brendan McMahan, and Swaroop Ramaswamy · 2019
Cited alongside, same era.
Lower bounds for non-convex stochastic optimization
Yossi Arjevani, Yair Carmon, John C Duchi, Dylan J Foster, Nathan Srebro, and Blake Woodworth · 2019
Cited alongside, same era.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov · 2019
Cited alongside, same era.
Rényi differential privacy of the sampled gaussian mechanism
Ilya Mironov, Kunal Talwar, and Li Zhang · 2019
Cited alongside, same era.
Differentially private learning needs better features (or much more data)
Florian Tramèr and Dan Boneh · 2020
Later among the works it cites.
Gradient perturbation is underrated for differentially private convex optimization
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie-Yan Liu · 2020
Later among the works it cites.
Improved analysis of clipping algorithms for non-convex optimization
Bohang Zhang, Jikai Jin, Cong Fang, and Liwei Wang · 2020
Later among the works it cites.
Why gradient clipping accelerates training: A theoretical justification for adaptivity
Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie · 2020
Later among the works it cites.
Private adaptive gradient methods for convex optimization
Hilal Asi, John C. Duchi, Alireza Fallah, Omid Javidbakht, and Kunal Talwar · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Adaclip: Adaptive clipping for private sgd
Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X Yu, Sashank J Reddi, and Sanjiv Kumar · 2019
Cited alongside, same era.
Differentially private empirical risk minimization with non-convex loss functions
Di Wang, Changyou Chen, and Jinhui Xu · 2019
Cited alongside, same era.
Subsampled rényi differential privacy and analytical moments accountant
Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan · 2019
Cited alongside, same era.
Reducing BERT pre-training time from 3 days to 76 minutes
Yang You, Jing Li, Jonathan Hseu, Xiaodan Song, James Demmel, and Cho-Jui Hsieh · 2019
Cited alongside, same era.
Deep learning with gaussian differential privacy
Zhiqi Bu, Jinshuo Dong, Qi Long, and Weijie J Su · 2020
Cited alongside, same era.
Extracting training data from large language models
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al · 2020
Cited alongside, same era.
Understanding gradient clipping in private sgd: A geometric perspective
Xiangyi Chen, Steven Z Wu, and Mingyi Hong · 2020
Cited alongside, same era.
Zhiqi Bu, Hua Wang, Qi Long, and Weijie J Su · 2021
Later among the works it cites.
High-probability bounds for non-convex stochastic optimization with heavy tails
Ashok Cutkosky and Harsh Mehta · 2021
Later among the works it cites.
Dp-normfedavg: Normalizing client updates for privacy-preserving federated learning
Rudrajit Das, Abolfazl Hashemi, Sujay Sanghavi, and Inderjit S Dhillon · 2021
Later among the works it cites.
Numerical composition of differential privacy
Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz · 2021
Later among the works it cites.
Non-convex distributionally robust optimization: Non-asymptotic analysis
Jikai Jin, Bohang Zhang, Haiyang Wang, and Liwei Wang · 2021
Later among the works it cites.
Large language models can be strong differentially private learners
Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto · 2021
Later among the works it cites.
The role of adaptive optimizers for honest private hyperparameter selection
Shubhankar Mohapatra, Sajin Sasy, Xi He, Gautam Kamath, and Om Thakkar · 2021
Later among the works it cites.
Hyperparameter tuning with renyi differential privacy
Nicolas Papernot and Thomas Steinke · 2021
Later among the works it cites.
Understanding clipping for federated learning: Convergence and client-level differential privacy
Xinwei Zhang, Xiangyi Chen, Mingyi Hong, Zhiwei Steven Wu, and Jinfeng Yi · 2021
Later among the works it cites.
On the convergence and improvement of stochastic normalized gradient descent
Shen-Yi Zhao, Yin-Peng Xie, and Wu-Jun Li · 2021
Later among the works it cites.
Differentially private sgd with non-smooth losses
Puyu Wang, Yunwen Lei, Yiming Ying, and Hai Zhang · 2022
Closest in time.