Fetching the paper…
Reading the bibliography…
In the privacy-utility tradeoff of a model trained on benchmark language and vision tasks, remarkable improvements have been widely reported with the use of pretraining on publicly available data.
The influence of pattern similarity and transfer learning upon training of a base perceptron b2
Stevo Bozinovski and Ante Fulgosi · 1976
Earlier work this paper cites.
A large-deviation inequality for vector-valued martingales
Thomas P. Hayes · 2003
Earlier work this paper cites.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith · 2006
Earlier work this paper cites.
Stochastic gradient descent with differentially private updates
Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate · 2013
Earlier work this paper cites.
Private empirical risk minimization: Efficient algorithms and tight error bounds
Raef Bassily, Adam Smith, and Abhradeep Thakurta · 2014
Earlier work this paper cites.
Cnn features off-the-shelf: an astounding baseline for recognition
Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson · 2014
Earlier work this paper cites.
Librispeech: an asr corpus based on public domain audio books
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Martín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
Train faster, generalize better: Stability of stochastic gradient descent
Moritz Hardt, Benjamin Recht, and Yoram Singer · 2016
Earlier work this paper cites.
The composition theorem for differential privacy
Peter Kairouz, Sewoong Oh, and Pramod Viswanath · 2017
Earlier work this paper cites.
The e2e dataset: New challenges for end-to-end generation
Jekaterina Novikova, Ondřej Dušek, and Verena Rieser · 2017
Earlier work this paper cites.
Tight lower bounds for differentially private selection
Thomas Steinke and Jonathan Ullman · 2017
Earlier work this paper cites.
Concentrated differentially private gradient descent with adaptive per-iteration privacy budget
Jaewoo Lee and Daniel Kifer · 2018
Earlier work this paper cites.
Limits of private learning with access to public data
Noga Alon, Raef Bassily, and Shay Moran · 2019
Earlier work this paper cites.
Private stochastic convex optimization with optimal rates
Raef Bassily, Vitaly Feldman, Kunal Talwar, and Abhradeep Guha Thakurta · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
The power of the hybrid model for mean estimation
Brendan Avent, Yatharth Dubey, and Aleksandra Korolova · 2020
Earlier work this paper cites.
Private query release assisted by public data
Raef Bassily, Albert Cheu, Shay Moran, Aleksandar Nikolov, Jonathan Ullman, and Steven Wu · 2020
Earlier work this paper cites.
Stability of stochastic gradient descent on nonsmooth convex losses
Raef Bassily, Vitaly Feldman, Cristóbal Guzmán, and Kunal Talwar · 2020
Cited alongside, same era.
Private stochastic convex optimization: Optimal rates in linear time
Vitaly Feldman, Tomer Koren, and Kunal Talwar · 2020
Cited alongside, same era.
Conformer: Convolution-augmented transformer for speech recognition
Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al · 2020
Cited alongside, same era.
Fast dimension independent private adagrad on publicly estimated subspaces
Peter Kairouz, Mónica Ribero, Keith Rush, and Abhradeep Thakurta · 2020
Cited alongside, same era.
Differentially private language models benefit from public pre-training
Gavin Kerrigan, Dylan Slack, and Jens Tuyls · 2020
Cited alongside, same era.
Private estimation with public data
Alex Bie, Gautam Kamath, and Vikrant Singhal · 2022
Later among the works it cites.
Differentially private bias-term only fine-tuning of foundation models
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis · 2022
Later among the works it cites.
Differentially private optimization on large model at small cost
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis · 2022
Later among the works it cites.
Unlocking high-accuracy differentially private image classification through scale
Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle · 2022
Later among the works it cites.
Mixed differential privacy in computer vision
Aditya Golatkar, Alessandro Achille, Yu-Xiang Wang, Aaron Roth, Michael Kearns, and Stefano Soatto · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, et al · 2020
Cited alongside, same era.
Differentially private learning needs better features (or much more data)
Florian Tramer and Dan Boneh · 2020
Cited alongside, same era.
Private adaptive gradient methods for convex optimization
Hilal Asi, John Duchi, Alireza Fallah, Omid Javidbakht, and Kunal Talwar · 2021
Cited alongside, same era.
Adapting to function difficulty and growth conditions in private optimization
Hilal Asi, Daniel Asher Nathan Levy, and John Duchi · 2021
Cited alongside, same era.
On the opportunities and risks of foundation models
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al · 2021
Cited alongside, same era.
Private non-smooth erm and sco in subquadratic steps
Janardhan Kulkarni, Yin Tat Lee, and Daogao Liu · 2021
Cited alongside, same era.
Leveraging public data for practical private query release
Terrance Liu, Giuseppe Vietri, Thomas Steinke, Jonathan Ullman, and Steven Wu · 2021
Cited alongside, same era.
Private convex optimization via exponential mechanism
Sivakanth Gopi, Yin Tat Lee, and Daogao Liu · 2022
Later among the works it cites.
On the universality of langevin diffusion for private euclidean (convex) optimization, 2022
Arun Ganesh, Abhradeep Thakurta, and Jalaj Upadhyay · 2022
Later among the works it cites.
Submix: Practical private prediction for large-scale language models
Antonio Ginart, Laurens van der Maaten, James Zou, and Chuan Guo · 2022
Later among the works it cites.
Exploring the limits of differentially private deep learning with group-wise clipping
Jiyan He, Xuechen Li, Da Yu, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, and Jiang Bian · 2022
Later among the works it cites.
Dynamic privacy budget allocation improves data efficiency of differentially private gradient descent
Junyuan Hong, Zhangyang Wang, and Jiayu Zhou · 2022
Later among the works it cites.
Dp 2 -vae: Differentially private pre-trained variational autoencoders
Dihong Jiang, Guojun Zhang, Mahdi Karami, Xi Chen, Yunfeng Shao, and Yaoliang Yu · 2022
Later among the works it cites.
Toward training at imagenet scale with differential privacy
Alexey Kurakin, Steve Chien, Shuang Song, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta · 2022
Later among the works it cites.
When does differentially private learning not suffer in high dimensions?
Xuechen Li, Daogao Liu, Tatsunori Hashimoto, Huseyin A Inan, Janardhan Kulkarni, YinTat Lee, and Abhradeep Guha Thakurta · 2022
Later among the works it cites.
Large language models can be strong differentially private learners
Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto · 2022
Later among the works it cites.
Effectively using public data in privacy preserving machine learning, 2022
Milad Nasr, Saeed Mahloujifar, Xinyu Tang, Prateek Mittal, and Amir Houmansadr · 2022
Later among the works it cites.
Considerations for differentially private learning with large-scale public pretraining
Florian Tramèr, Gautam Kamath, and Nicholas Carlini · 2022
Later among the works it cites.
Differentially private fine-tuning of language models
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al · 2022
Later among the works it cites.