Fetching the paper…
Reading the bibliography…
Differentially private deep learning has recently witnessed advances in computational efficiency and privacy-utility trade-off.
Stochastic gradient descent with differentially private updates
Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate · 2013
Earlier work this paper cites.
Private empirical risk minimization: Efficient algorithms and tight error bounds
Raef Bassily, Adam Smith, and Abhradeep Thakurta · 2014
Earlier work this paper cites.
The algorithmic foundations of differential privacy
Cynthia Dwork, Aaron Roth, et al · 2014
Earlier work this paper cites.
Efficient per-example gradient computations
Ian Goodfellow · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
Deep models under the gan: information leakage from collaborative deep learning
Briland Hitaj, Giuseppe Ateniese, and Fernando Pérez-Cruz · 2017
Earlier work this paper cites.
Rényi differential privacy
Ilya Mironov · 2017
Earlier work this paper cites.
The E2E dataset: New challenges for end-to-end generation
Jekaterina Novikova, Ondřej Dušek, and Verena Rieser · 2017
Earlier work this paper cites.
Semi-supervised knowledge transfer for deep learning from private training data
Nicolas Papernot, Martín Abadi, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar · 2017
Earlier work this paper cites.
Membership inference attacks against machine learning models
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov · 2017
Earlier work this paper cites.
Compiling machine learning programs via high-level tracing
Roy Frostig, Matthew James Johnson, and Chris Leary · 2018
Earlier work this paper cites.
Glue: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman · 2018
Earlier work this paper cites.
Differentially private learning with adaptive clipping
Galen Andrew, Om Thakkar, H Brendan McMahan, and Swaroop Ramaswamy · 2019
Earlier work this paper cites.
Assessing differentially private deep learning with membership inference
Daniel Bernau, Philip-William Grassal, Jonas Robl, and Florian Kerschbaum · 2019
Earlier work this paper cites.
The secret sharer: Evaluating and testing unintended memorization in neural networks
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song · 2019
Earlier work this paper cites.
Samsum corpus: A human-annotated dialogue dataset for abstractive summarization
Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer · 2019
Earlier work this paper cites.
Gpipe: Efficient training of giant neural networks using pipeline parallelism
Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, et al · 2019
Earlier work this paper cites.
Roberta: A robustly optimized BERT pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov · 2019
Earlier work this paper cites.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al · 2019
Earlier work this paper cites.
AdaCliP: Adaptive clipping for private SGD
Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X Yu, Sashank J Reddi, and Sanjiv Kumar · 2019
Earlier work this paper cites.
Privacy risks of securing machine learning models against adversarial examples
Liwei Song, Reza Shokri, and Prateek Mittal · 2019
Cited alongside, same era.
Deep leakage from gradients
Ligeng Zhu, Zhijian Liu, and Song Han · 2019
Cited alongside, same era.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Cited alongside, same era.
Extracting training data from large language models
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al · 2020
Cited alongside, same era.
Fast dimension independent private adagrad on publicly estimated subspaces
Peter Kairouz, Mónica Ribero, Keith Rush, and Abhradeep Thakurta · 2020
Cited alongside, same era.
Multitask prompted training enables zero-shot task generalization
Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al · 2021
Later among the works it cites.
Manuel Senge, Timour Igamberdiev, and Ivan Habernal · 2021
Later among the works it cites.
Enabling fast differentially private SGD via just-in-time compilation and vectorization
Pranav Subramani, Nicholas Vadivelu, and Gautam Kamath · 2021
Later among the works it cites.
SeqPATE: Differentially Private Text Generation via Knowledge Distillation
Zhiliang Tian, Yingxiu Zhao, Ziyue Huang, Yu-Xiang Wang, Nevin Zhang, and He He · 2021
Later among the works it cites.
Differentially private learning needs better features (or much more data)
Florian Tramèr and Dan Boneh · 2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Differentially private language models benefit from public pre-training
Gavin Kerrigan, Dylan Slack, and Jens Tuyls · 2020
Cited alongside, same era.
Dart: Open-domain structured data record to text generation
Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, et al · 2020
Cited alongside, same era.
Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He · 2020
Cited alongside, same era.
Characterizing private clipped gradient descent on convex generalized linear problems
Shuang Song, Om Thakkar, and Abhradeep Thakurta · 2020
Cited alongside, same era.
Private-knn: Practical differential privacy for computer vision
Yuqing Zhu, Xiang Yu, Manmohan Chandraker, and Yu-Xiang Wang · 2020
Cited alongside, same era.
Large-scale differentially private BERT
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Manurangsi · 2021
Cited alongside, same era.
Private adaptive gradient methods for convex optimization
Hilal Asi, John C. Duchi, Alireza Fallah, Omid Javidbakht, and Kunal Talwar · 2021
Cited alongside, same era.
Later among the works it cites.
Finetuned language models are zero-shot learners
Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le · 2021
Later among the works it cites.
Opacus: User-friendly differential privacy library in pytorch
Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, et al · 2021
Later among the works it cites.
Bypassing the ambient dimension: Private {sgd} with gradient subspace identification
Yingxue Zhou, Steven Wu, and Arindam Banerjee · 2021
Later among the works it cites.
Reconstructing training data with informed adversaries
Borja Balle, Giovanni Cherubin, and Jamie Hayes · 2022
Closest in time.
Scalable and efficient training of large convolutional neural networks with differential privacy
Zhiqi Bu, Jialin Mao, and Shiyun Xu · 2022
Closest in time.
Membership inference attacks from first principles
Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer · 2022
Closest in time.
Unlocking high-accuracy differentially private image classification through scale
Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle · 2022
Closest in time.
An efficient DP-SGD mechanism for large scale NLU models
Christophe Dupuy, Radhika Arava, Rahul Gupta, and Anna Rumshisky · 2022
Closest in time.
Mixed differential privacy in computer vision
Aditya Golatkar, Alessandro Achille, Yu-Xiang Wang, Aaron Roth, Michael Kearns, and Stefano Soatto · 2022
Closest in time.
Training compute-optimal large language models
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al · 2022
Closest in time.
Toward training at ImageNet scale with differential privacy
Alexey Kurakin, Steve Chien, Shuang Song, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta · 2022
Closest in time.
Dimension independent generalization of DP-SGD for overparameterized smooth convex optimization
Yi-An Ma, Teodor Vanislavov Marinov, and Tong Zhang · 2022
Closest in time.
Large scale transfer learning for differentially private image classification
Harsh Mehta, Abhradeep Thakurta, Alexey Kurakin, and Ashok Cutkosky · 2022
Closest in time.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Closest in time.
Differentially private fine-tuning of language models
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang · 2022
Closest in time.
Introducing pytorch fully sharded data parallel (fsdp) api
Yanli Zhao, Rohan Varma, Chien-Chin Huang, Shen Li, Min Xu, and Alban Desmaison · 2022
Closest in time.