Fetching the paper…
Reading the bibliography…
Fine-tuning is a common and effective method for tailoring large language models (LLMs) to specialized tasks and applications.
Testing Statistical Hypotheses , volume 3
E. L. Lehmann, J. P. Romano, and G. Casella · 1986
Earlier work this paper cites.
Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping
R. Caruana, S. Lawrence, and C. Giles · 2000
Earlier work this paper cites.
Introducing the enron corpus
B. Klimt and Y. Yang · 2004
Earlier work this paper cites.
Calibrating Noise to Sensitivity in Private Data Analysis
C. Dwork, F. McSherry, K. Nissim, and A. Smith · 2006
Earlier work this paper cites.
Authorship attribution and verification with many authors and limited data
K. Luyckx and W. Daelemans · 2008
Earlier work this paper cites.
The effect of author set size and data size in authorship attribution
K. Luyckx and W. Daelemans · 2010
Earlier work this paper cites.
On the difficulty of training recurrent neural networks
R. Pascanu, T. Mikolov, and Y. Bengio · 2013
Earlier work this paper cites.
The Composition Theorem for Differential Privacy
P. Kairouz, S. Oh, and P. Viswanath · 2015
Earlier work this paper cites.
Adam: A Method for Stochastic Optimization
D. P. Kingma and J. Ba · 2015
Earlier work this paper cites.
Deep learning with differential privacy
M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang · 2016
Earlier work this paper cites.
news-please: A generic news crawler and extractor
F. Hamborg, N. Meuschke, C. Breitinger, and B. Gipp · 2017
Earlier work this paper cites.
Membership inference attacks against machine learning models
R. Shokri, M. Stronati, C. Song, and V. Shmatikov · 2017
Earlier work this paper cites.
Property Inference Attacks on Fully Connected Neural Networks Using Permutation Invariant Representations
K. Ganju, Q. Wang, W. Yang, C. A. Gunter, and N. Borisov · 2018
Earlier work this paper cites.
Learning differentially private recurrent language models
H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang · 2018
Earlier work this paper cites.
Privacy risk in machine learning: Analyzing the connection to overfitting
S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha · 2018
Earlier work this paper cites.
Gmail smart compose: Real-time assisted writing
M. X. Chen, B. N. Lee, G. Bansal, Y. Cao, S. Zhang, J. Lu, J. Tsay, Y. Wang, A. M. Dai, Z. Chen, T. Sohn, and Y. Wu · 2019
Earlier work this paper cites.
On the use of arxiv as a dataset
C. B. Clement, M. Bierbaum, K. P. O’Keeffe, and A. A. Alemi · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever · 2019
Earlier work this paper cites.
White-box vs black-box: Bayes optimal strategies for membership inference
A. Sablayrolles, M. Douze, C. Schmid, Y. Ollivier, and H. Jégou · 2019
Earlier work this paper cites.
Auditing data provenance in text-generation models
C. Song and V. Shmatikov · 2019
Earlier work this paper cites.
Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
Z. Wang, M. Song, Z. Zhang, Y. Song, Q. Wang, and H. Qi · 2019
Earlier work this paper cites.
The pushshift reddit dataset
J. Baumgartner, S. Zannettou, B. Keegan, M. Squire, and J. Blackburn · 2020
Earlier work this paper cites.
The pile: An 800gb dataset of diverse text for language modeling
L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al · 2020
Cited alongside, same era.
Training production language models without memorizing user data
S. Ramaswamy, O. Thakkar, R. Mathews, G. Andrew, H. B. McMahan, and F. Beaufays · 2020
Cited alongside, same era.
Analyzing User-Level Privacy Attack Against Federated Learning
M. Song, Z. Wang, Z. Zhang, Y. Song, Q. Wang, J. Ren, and H. Qi · 2020
Cited alongside, same era.
GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow, Mar. 2021
S. Black, L. Gao, P. Wang, C. Leahy, and S. Biderman · 2021
Cited alongside, same era.
Extracting training data from large language models
N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, Ú. Erlingsson, A. Oprea, and C. Raffel · 2021
On the importance of difficulty calibration in membership inference attacks
L. Watson, C. Guo, G. Cormode, and A. Sablayrolles · 2022
Later among the works it cites.
Enhanced membership inference attacks against machine learning models
J. Ye, A. Maddi, S. K. Murakonda, V. Bindschaedler, and R. Shokri · 2022
Later among the works it cites.
Opt: Open pre-trained transformer language models
S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, T. Mihaylov, M. Ott, S. Shleifer, K. Shuster, D. Simig, P. S. Koura, A. Sridhar, T. Wang, and L. Zettlemoyer · 2022
Later among the works it cites.
R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, et al · 2023
Closest in time.
Emergent and Predictable Memorization in Large Language Models
S. Biderman, U. S. Prashanth, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, and E. Raf · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Evaluating large language models trained on code
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba · 2021
Cited alongside, same era.
Label-only membership inference attacks
C. A. Choquette-Choo, F. Tramer, N. Carlini, and N. Papernot · 2021
Cited alongside, same era.
Training data leakage analysis in language models
H. A. Inan, O. Ramadan, L. Wutschitz, D. Jones, V. Rühle, J. Withers, and R. Sim · 2021
Cited alongside, same era.
Practical and private (deep) learning without sampling or shuffling
P. Kairouz, B. McMahan, S. Song, O. Thakkar, A. Thakurta, and Z. Xu · 2021
Cited alongside, same era.
Learning with user-level privacy
D. A. N. Levy, Z. Sun, K. Amin, S. Kale, A. Kulesza, M. Mohri, and A. T. Suresh · 2021
Cited alongside, same era.
The Audio Auditor: User-Level Membership Inference in Internet of Things Voice Services
Y. Miao, M. Xue, C. Chen, L. Pan, J. Zhang, B. Z. H. Zhao, D. Kaafar, and Y. Xiang · 2021
Cited alongside, same era.
Adaptive Federated Optimization
S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konečný, S. Kumar, and H. B. McMahan · 2021
Cited alongside, same era.
Closest in time.
Quantifying memorization across neural language models
N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang · 2023
Closest in time.
Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning
Z. Charles, N. Mitchell, K. Pillutla, M. Reneer, and Z. Garrett · 2023
Closest in time.
FACE-AUDITOR: Data auditing in facial recognition systems
M. Chen, Z. Zhang, T. Wang, M. Backes, and Y. Zhang · 2023
Closest in time.
Privacy side channels in machine learning systems
E. Debenedetti, G. Severi, N. Carlini, C. A. Choquette-Choo, M. Jagielski, M. Nasr, E. Wallace, and F. Tramèr · 2023
Closest in time.
Distribution Inference Risks: Identifying and Mitigating Sources of Leakage
V. Hartmann, L. Meynent, M. Peyrard, D. Dimitriadis, S. Tople, and R. West · 2023
Closest in time.
Preventing generation of verbatim memorization in language models gives a false sense of privacy
D. Ippolito, F. Tramer, M. Nasr, C. Zhang, M. Jagielski, K. Lee, C. Choquette Choo, and N. Carlini · 2023
Closest in time.
Madlad-400: A multilingual and document-level large audited dataset
S. Kudugunta, I. Caswell, B. Zhang, X. Garcia, C. A. Choquette-Choo, K. Lee, D. Xin, A. Kusupati, R. Stella, A. Bapna, et al · 2023
Closest in time.
Analyzing leakage of personally identifiable information in language models
N. Lukas, A. Salem, R. Sim, S. Tople, L. Wutschitz, and S. Zanella-Beguelin · 2023
Closest in time.
Membership inference attacks against language models via neighbourhood comparison
J. Mattern, F. Mireshghallah, Z. Jin, B. Schoelkopf, M. Sachan, and T. Berg-Kirkpatrick · 2023
Closest in time.
Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation
M. Mosbach, T. Pimentel, S. Ravfogel, D. Klakow, and Y. Elazar · 2023
Closest in time.
Scalable extraction of training data from (production) language models
M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee · 2023
Closest in time.
CodexLeaks: Privacy leaks from code generation language models in GitHub copilot
L. Niu, S. Mirza, Z. Maradni, and C. Pöpper · 2023
Closest in time.
Adversarial machine learning: A taxonomy and terminology of attacks and mitigations
A. Oprea and A. Vassilev · 2023
Closest in time.
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy
N. Ponomareva, H. Hazimeh, A. Kurakin, Z. Xu, C. Denison, H. B. McMahan, S. Vassilvitskii, S. Chien, and A. G. Thakurta · 2023
Closest in time.
Did ChatGPT Cheat on Your Test?
O. Sainz, J. A. Campos, I. García-Ferrero, J. Etxaniz, and E. Agirre · 2023
Closest in time.
Federated learning of gboard language models with differential privacy
Z. Xu, Y. Zhang, G. Andrew, C. Choquette, P. Kairouz, B. Mcmahan, J. Rosenstock, and Y. Zhang · 2023
Closest in time.