Fetching the paper…
Reading the bibliography…
Text prediction models, when used in applications like email clients or word processors, must protect user data privacy and adhere to model size constraints.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith · 2006
Earlier work this paper cites.
Stochastic gradient descent with differentially private updates
Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate · 2013
Earlier work this paper cites.
Differentially private empirical risk minimization: Efficient algorithms and tight error bounds
Raef Bassily, Adam Smith, and Abhradeep Thakurta · 2014
Earlier work this paper cites.
Enron email dataset, 2015
William Cohen · 2015
Earlier work this paper cites.
Learning both weights and connections for efficient neural network
Song Han, Jeff Pool, John Tran, and William Dally · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
On calibration of modern neural networks
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger · 2017
Earlier work this paper cites.
Semi-supervised knowledge transfer for deep learning from private training data
Nicolas Papernot, Martín shokri, Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar · 2017
Earlier work this paper cites.
Membership inference attacks against machine learning models
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov · 2017
Earlier work this paper cites.
Learning differentially private recurrent language models
H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training, 2018
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever · 2018
Earlier work this paper cites.
Glue: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman · 2018
Earlier work this paper cites.
Limits of private learning with access to public data
Noga Alon, Raef Bassily, and Shay Moran · 2019
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Earlier work this paper cites.
Openwebtext corpus
Aaron Gokaslan and Vanya Cohen · 2019
Earlier work this paper cites.
LOGAN: membership inference attacks against generative models
Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro · 2019
Earlier work this paper cites.
Private selection from private candidates
Jingcheng Liu and Kunal Talwar · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf · 2019
Earlier work this paper cites.
Language models are few-shot learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Earlier work this paper cites.
Selection via proxy: Efficient data selection for deep learning
Cody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia · 2020
Earlier work this paper cites.
Don’t stop pretraining: Adapt language models to domains and tasks
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith · 2020
Earlier work this paper cites.
Scaling laws for neural language models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei · 2020
Earlier work this paper cites.
Differentially private language models benefit from public pre-training
Gavin Kerrigan, Dylan Slack, and Jens Tuyls · 2020
Earlier work this paper cites.
Computing tight differential privacy guarantees using FFT
Antti Koskela, Joonas Jälkö, and Antti Honkela · 2020
Earlier work this paper cites.
Assistive ai makes replying easier, 2020
Microsoft · 2020
Earlier work this paper cites.
Deep domain adaptation with differential privacy
Qian Wang, Zixi Li, Qin Zou, Lingchen Zhao, and Song Wang · 2020
Earlier work this paper cites.
Private-knn: Practical differential privacy for computer vision
Yuqing Zhu, Xiang Yu, Manmohan Chandraker, and Yu-Xiang Wang · 2020
Cited alongside, same era.
Fast and memory efficient differentially private-sgd via jl projections
Zhiqi Bu, Sivakanth Gopi, Janardhan Kulkarni, Yin Tat Lee, Hanwen Shen, and Uthaipon Tantipongpipat · 2021
Cited alongside, same era.
Extracting training data from large language models
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al · 2021
Cited alongside, same era.
Label-only membership inference attacks
Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot · 2021
Cited alongside, same era.
Numerical composition of differential privacy
Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz · 2021
Cited alongside, same era.
(nearly) dimension independent private ERM with adagrad rates via publicly estimated subspaces
Prioritized training on points that are learnable, worth learning, and not yet learnt
Sören Mindermann, Jan Markus Brauner, Muhammed Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen, Aidan N. Gomez, Adrien Morisot, Sebastian Farquhar, and Yarin Gal · 2022
Later among the works it cites.
Differentially private model compression
Fatemehsadat Mireshghallah, Arturs Backurs, Huseyin A Inan, Lukas Wutschitz, and Janardhan Kulkarni · 2022
Later among the works it cites.
The role of adaptive optimizers for honest private hyperparameter selection
Shubhankar Mohapatra, Sajin Sasy, Xi He, Gautam Kamath, and Om Thakkar · 2022
Later among the works it cites.
Dp-raft: A differentially private recipe for accelerated fine-tuning
Ashwinee Panda, Xinyu Tang, Vikash Sehwag, Saeed Mahloujifar, and Prateek Mittal · 2022
Later among the works it cites.
Hyperparameter tuning with renyi differential privacy
Nicolas Papernot and Thomas Steinke · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Peter Kairouz, Mónica Ribero, Keith Rush, and Abhradeep Thakurta · 2021
Cited alongside, same era.
Scaling up differentially private deep learning with fast per-example gradient clipping
Jaewoo Lee and Daniel Kifer · 2021
Cited alongside, same era.
Scalable differential privacy with sparse network finetuning
Zelun Luo, Daniel J. Wu, Ehsan Adeli, and Li Fei-Fei · 2021
Cited alongside, same era.
Enabling fast differentially private sgd via just-in-time compilation and vectorization
Pranav Subramani, Nicholas Vadivelu, and Gautam Kamath · 2021
Cited alongside, same era.
Differentially private learning needs better features (or much more data)
Florian Tramèr and Dan Boneh · 2021
Cited alongside, same era.
Tent: Fully test-time adaptation by entropy minimization
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell · 2021
Cited alongside, same era.
Large scale private learning via low-rank reparametrization
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie-Yan Liu · 2021
Cited alongside, same era.
Later among the works it cites.
Tan without a burn: Scaling laws of dp-sgd
Tom Sander, Pierre Stock, and Alexandre Sablayrolles · 2022
Later among the works it cites.
Beyond neural scaling laws: beating power law scaling via data pruning
Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos · 2022
Later among the works it cites.
Composition of differential privacy & privacy amplification by subsampling
Thomas Steinke · 2022
Later among the works it cites.
Considerations for differentially private learning with large-scale public pretraining
Florian Tramèr, Gautam Kamath, and Nicholas Carlini · 2022
Later among the works it cites.
Debugging differential privacy: A case study for privacy auditing
Florian Tramer, Andreas Terzis, Thomas Steinke, Shuang Song, Matthew Jagielski, and Nicholas Carlini · 2022
Later among the works it cites.
Structured pruning learns compact and accurate models
Mengzhou Xia, Zexuan Zhong, and Danqi Chen · 2022
Later among the works it cites.
Public data assisted differential private deep learning
Jiaxi Yang and Xiang Cheng · 2022
Later among the works it cites.
Differentially private fine-tuning of language models
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al · 2022
Later among the works it cites.
A closer look at the calibration of differentially private learners
Hanlin Zhang, Xuechen Li, Prithviraj Sen, Salim Roukos, and Tatsunori Hashimoto · 2022
Later among the works it cites.
Transfer adaptation learning: A decade survey
L Zhang and X Gao · 2022
Later among the works it cites.
Principled approaches for private adaptation from a public source
Raef Bassily, Mehryar Mohri, and Ananda Theertha Suresh · 2023
Closest in time.
Quantifying memorization across neural language models
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang · 2023
Closest in time.
Why is public pretraining necessary for private model training?
Arun Ganesh, Mahdi Haghifam, Milad Nasr, Sewoong Oh, Thomas Steinke, Om Thakkar, Abhradeep Thakurta, and Lun Wang · 2023
Closest in time.
Choosing public datasets for private machine learning via gradient subspace distance
Xin Gu, Gautam Kamath, and Zhiwei Steven Wu · 2023
Closest in time.
Exploring the limits of differentially private deep learning with group-wise clipping
Jiyan He, Xuechen Li, Da Yu, Huishuai Zhang, Janardhan Kulkarni, Yin Tat Lee, Arturs Backurs, Nenghai Yu, and Jiang Bian · 2023
Closest in time.
Privately customizing prefinetuning to better match user data in federated learning
Charlie Hou, Hongyuan Zhan, Akshat Shrivastava, Sid Wang, Sasha Livshits, Giulia Fanti, and Daniel Lazar · 2023
Closest in time.
A data-based perspective on transfer learning
Saachi Jain, Hadi Salman, Alaa Khaddaj, Eric Wong, Sung Min Park, and Aleksander Madry · 2023
Closest in time.
Harnessing large-language models to generate private synthetic text
Alexey Kurakin, Natalia Ponomareva, Umar Syed, Liam MacDermed, and Andreas Terzis · 2023
Closest in time.
Membership inference attacks against diffusion models
Tomoya Matsumoto, Takayuki Miura, and Naoto Yanai · 2023
Closest in time.
Data selection for language models via importance resampling
Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang · 2023
Closest in time.
Federated learning of gboard language models with differential privacy
Zheng Xu, Yanxiang Zhang, Galen Andrew, Christopher A Choquette-Choo, Peter Kairouz, H Brendan McMahan, Jesse Rosenstock, and Yuanbo Zhang · 2023
Closest in time.
Unsupervised domain adaptation with differentially private gradient projection
Maobo Zheng, Xiaojian Zhang, Xuebin Ma, et al · 2023
Closest in time.