Fetching the paper…
Reading the bibliography…
On-device training is currently the most common approach for training machine learning (ML) models on private, distributed user data.
Roberta: A robustly optimized BERT pretraining approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V · 1907
Earlier work this paper cites.
Sentence-bert: Sentence embeddings using siamese bert-networks
Reimers, N. and Gurevych, I · 1908
Earlier work this paper cites.
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L · 1910
Earlier work this paper cites.
Differential privacy
Dwork, C · 2006
Earlier work this paper cites.
Analyze gauss: optimal bounds for privacy-preserving principal component analysis
Dwork, C., Talwar, K., Thakurta, A., and Zhang, L · 2014
Earlier work this paper cites.
Deep learning with differential privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L · 2016
Earlier work this paper cites.
Practical secure aggregation for federated learning on user-held data
Bonawitz, K. A., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., and Seth, K · 2016
Earlier work this paper cites.
Semi-supervised knowledge transfer for deep learning from private training data
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I. J., and Talwar, K · 2016
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S · 2017
Earlier work this paper cites.
cpsgd: Communication-efficient and differentially-private distributed sgd
Agarwal, N., Suresh, A. T., Yu, F. X. X., Kumar, S., and McMahan, B · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Earlier work this paper cites.
Scalable private learning with pate
Papernot, N., Song, S., Mironov, I., Raghunathan, A., Talwar, K., and Erlingsson, Ú · 2018
Earlier work this paper cites.
TensorFlow Federated, December 2018
Tensorflow · 2018
Earlier work this paper cites.
Wang, T., Zhu, J.-Y., Torralba, A., and Efros, A. A · 2018
Earlier work this paper cites.
Generative models for effective ml on private, decentralized datasets
Augenstein, S., McMahan, H. B., Ramage, D., Ramaswamy, S., Kairouz, P., Chen, M., Mathews, R., et al · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2019
Earlier work this paper cites.
Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., and Wolf, T · 2019
Earlier work this paper cites.
Dp-cgan: Differentially private synthetic data and label generation
Torkzadehmahani, R., Kairouz, P., and Paten, B · 2019
Earlier work this paper cites.
Distributed distillation for on-device learning
Bistritz, I., Mann, A., and Bambos, N · 2020
Earlier work this paper cites.
Dimension independence in unconstrained private erm via adaptive preconditioning
Kairouz, P., Ribero, M., Rush, K., and Thakurta, A · 2020
Earlier work this paper cites.
Scaffold: Stochastic controlled averaging for federated learning
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A. T · 2020
Earlier work this paper cites.
Differentially private language models benefit from public pre-training
Kerrigan, G., Slack, D., and Tuyls, J · 2020
Earlier work this paper cites.
Federated optimization in heterogeneous networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V · 2020
Earlier work this paper cites.
Ensemble distillation for robust model fusion in federated learning
Lin, T., Kong, L., Stich, S. U., and Jaggi, M · 2020
Earlier work this paper cites.
Neunhoeffer, M., Wu, Z. S., and Dwork, C · 2020
Earlier work this paper cites.
Training production language models without memorizing user data
Ramaswamy, S., Thakkar, O., Mathews, R., Andrew, G., McMahan, H. B., and Beaufays, F · 2020
Earlier work this paper cites.
Differentially private learning needs better features (or much more data)
Tramèr, F. and Boneh, D · 2020
Earlier work this paper cites.
Tackling the objective inconsistency problem in heterogeneous federated optimization
Wang, J., Liu, Q., Liang, H., Joshi, G., and Poor, H. V · 2020
Earlier work this paper cites.
Dataset condensation with gradient matching
Zhao, B., Mopuri, K. R., and Bilen, H · 2020
Earlier work this paper cites.
Bypassing the ambient dimension: Private sgd with gradient subspace identification
Zhou, Y., Wu, Z. S., and Banerjee, A · 2020
Earlier work this paper cites.
The skellam mechanism for differentially private federated learning
Agarwal, N., Kairouz, P., and Liu, Z · 2021
Earlier work this paper cites.
Large-scale differentially private bert
Anil, R., Ghazi, B., Gupta, V., Kumar, R., and Manurangsi, P · 2021
Earlier work this paper cites.
Bagdasaryan, E., Kairouz, P., Mellem, S., Gascón, A., Bonawitz, K., Estrin, D., and Gruteser, M · 2021
Earlier work this paper cites.
Don’t generate me: Training differentially private generative models with sinkhorn divergence
Cao, T., Bie, A., Vahdat, A., Fidler, S., and Kreis, K · 2021
Earlier work this paper cites.
Personalized federated learning for heterogeneous clients with clustered knowledge transfer
Cho, Y. J., Wang, J., Chiruvolu, T., and Joshi, G · 2021
Earlier work this paper cites.
Shuffled model of federated learning: Privacy, accuracy and communication trade-offs
Girgis, A. M., Data, D., Diggavi, S., Kairouz, P., and Suresh, A. T · 2021
Cited alongside, same era.
Large language models can be strong differentially private learners
Li, X., Tramer, F., Liang, P., and Hashimoto, T · 2021
Cited alongside, same era.
On the privacy properties of gan-generated samples
Lin, Z., Sekar, V., and Fanti, G · 2021
Cited alongside, same era.
Privacy regularization: Joint privacy-utility optimization in language models
Mireshghallah, F., Inan, H. A., Hasegawa, M., Rühle, V., Berg-Kirkpatrick, T., and Sim, R · 2021
Cited alongside, same era.
A field guide to federated optimization
Wang, J., Charles, Z., Xu, Z., Joshi, G., McMahan, H. B., Al-Shedivat, M., Andrew, G., Avestimehr, S., Daly, K., Data, D., et al · 2021
Dp-forward: Fine-tuning and inference on language models with differential privacy in forward pass
Du, M., Yue, X., Chow, S. S., Wang, T., Huang, C., and Sun, H · 2023
Later among the works it cites.
Why is public pretraining necessary for private model training?
Ganesh, A., Haghifam, M., Nasr, M., Oh, S., Steinke, T., Thakkar, O., Thakurta, A. G., and Wang, L · 2023
Later among the works it cites.
Can 5th generation local training methods support client sampling? yes!
Grudzień, M., Malinovsky, G., and Richtárik, P · 2023
Later among the works it cites.
Choosing public datasets for private machine learning via gradient subspace distance
Gu, X., Kamath, G., and Wu, Z. S · 2023
Later among the works it cites.
Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Del Giorno, A., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G., Saarikivi, O., et al · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Communication-efficient federated learning via knowledge distillation
Wu, C., Wu, F., Lyu, L., Huang, Y., and Xie, X · 2021
Cited alongside, same era.
On a utilitarian approach to privacy preserving text generation
Xu, Z., Aggarwal, A., Feyisetan, O., and Teissier, N · 2021
Cited alongside, same era.
Opacus: User-friendly differential privacy library in PyTorch
Yousefpour, A., Shilov, I., Sablayrolles, A., Testuggine, D., Prasad, K., Malek, M., Nguyen, J., Ghosh, S., Bharadwaj, A., Zhao, J., Cormode, G., and Mironov, I · 2021
Cited alongside, same era.
What does it mean for a language model to preserve privacy?
Brown, H., Lee, K., Mireshghallah, F., Shokri, R., and Tramèr, F · 2022
Cited alongside, same era.
Fedadapter: Efficient federated learning for modern nlp
Cai, D., Wu, Y., Wang, S., Lin, F. X., and Xu, M · 2022
Cited alongside, same era.
Dataset distillation by matching training trajectories
Cazenavette, G., Wang, T., Torralba, A., Efros, A. A., and Zhu, J.-Y · 2022
Cited alongside, same era.
Federated select: A primitive for communication-and memory-efficient federated learning
Charles, Z., Bonawitz, K., Chiknavaryan, S., McMahan, B., et al · 2022
Cited alongside, same era.
Later among the works it cites.
Promptfl: Let federated participants cooperatively learn prompts instead of models-federated learning in age of foundation model
Guo, T., Guo, S., Wang, J., Tang, X., and Xu, W · 2023
Later among the works it cites.
Privately customizing prefinetuning to better match user data in federated learning
Hou, C., Zhan, H., Shrivastava, A., Wang, S., Livshits, S., Fanti, G., and Lazar, D · 2023
Later among the works it cites.
Harnessing large-language models to generate private synthetic text
Kurakin, A., Ponomareva, N., Syed, U., MacDermed, L., and Terzis, A · 2023
Later among the works it cites.
Efficient memory management for large language model serving with pagedattention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I · 2023
Later among the works it cites.
Privacy-preserving prompt tuning for large language model services
Li, Y., Tan, Z., and Liu, Y · 2023
Later among the works it cites.
Differentially private synthetic data via foundation model apis 1: Images
Lin, Z., Gopi, S., Kulkarni, J., Nori, H., and Yekhanin, S · 2023
Later among the works it cites.
Effectively using public data in privacy preserving machine learning
Nasr, M., Mahloujifar, S., Tang, X., Mittal, P., and Houmansadr, A · 2023
Later among the works it cites.
How to dp-fy ml: A practical guide to machine learning with differential privacy
Ponomareva, N., Hazimeh, H., Kurakin, A., Xu, Z., Denison, C. E., McMahan, H. B., Vassilvitskii, S., Chien, S., and Thakurta, A · 2023
Later among the works it cites.
Code llama: Open foundation models for code
Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Remez, T., Rapin, J., et al · 2023
Later among the works it cites.
Federated learning via decentralized dataset distillation in resource-constrained edge environments
Song, R., Liu, D., Chen, D. Z., Festag, A., Trinitis, C., Schulz, M., and Knoll, A · 2023
Later among the works it cites.
United states median country speeds october 2023
speedtest.net · 2023
Later among the works it cites.
Privacy-preserving in-context learning with differentially private few-shot generation
Tang, X., Shin, R., Inan, H. A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R · 2023
Later among the works it cites.
Stanford alpaca: An instruction-following llama model
Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B · 2023
Later among the works it cites.
Llama: Open and efficient foundation language models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al · 2023
Later among the works it cites.
Locally differentially private document generation using zero shot prompting
Utpala, S., Hooker, S., and Chen, P.-Y · 2023
Later among the works it cites.
Privacy-preserving in-context learning for large language models
Wu, T., Panda, A., Wang, J. T., and Mittal, P · 2023
Later among the works it cites.
Training large-vocabulary neural language models by private federated learning for resource-constrained devices
Xu, M., Song, C., Tian, Y., Agrawal, N., Granqvist, F., van Dalen, R., Zhang, X., Argueta, A., Han, S., Deng, Y., et al · 2023
Later among the works it cites.
Automatic clipping: Differentially private deep learning made easier and stronger
Bu, Z., Wang, Y.-X., Zha, S., and Karypis, G · 2024
Closest in time.
Delving into differentially private transformer
Ding, Y., Wu, X., Meng, Y., Luo, Y., Wang, H., and Pan, W · 2024
Closest in time.
Flocks of stochastic parrots: Differentially private prompt learning for large language models
Duan, H., Dziedzic, A., Papernot, N., and Boenisch, F · 2024
Closest in time.
A unified fast gradient clipping framework for dp-sgd
Kong, W. and Munoz Medina, A · 2024
Closest in time.
Reddit to give openai access to its data in licensing deal
Needleman, S · 2024
Closest in time.
PyTorch Mobile, 2024
PyTorch · 2024
Closest in time.
For data-guzzling ai companies, the internet is too small
Seetharaman, D · 2024
Closest in time.
Private fine-tuning of large language models with zeroth-order optimization
Tang, X., Panda, A., Nasr, M., Mahloujifar, S., and Mittal, P · 2024
Closest in time.
Exclusive: Reddit in ai content licensing deal with google
Tong, A., Wang, E., and Coulter, M · 2024
Closest in time.
Prompt public large language models to synthesize data for private on-device applications
Wu, S., Xu, Z., Zhang, Y., Zhang, Y., and Ramage, D · 2024
Closest in time.
Differentially private synthetic data via foundation model apis 2: Text
Xie, C., Lin, Z., Backurs, A., Gopi, S., Yu, D., Inan, H. A., Nori, H., Jiang, H., Zhang, H., Lee, Y. T., et al · 2024
Closest in time.
Machine unlearning of pre-trained large language models
Yao, J., Chien, E., Du, M., Niu, X., Wang, T., Cheng, Z., and Yue, X · 2024
Closest in time.
Privacy-preserving instructions for aligning large language models
Yu, D., Kairouz, P., Oh, S., and Xu, Z · 2024
Closest in time.