Fetching the paper…
Reading the bibliography…
How much information about training samples can be leaked through synthetic data generated by Large Language Models (LLMs)? Overlooking the subtleties of information flow in synthetic data generation pipelines can lead to a false sense of privacy.
RoBERTa: A robustly optimized BERT pretraining approach, 2019
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V · 1907
Earlier work this paper cites.
Recursive deep models for semantic compositionality over a sentiment treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C · 2013
Earlier work this paper cites.
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D · 2015
Earlier work this paper cites.
Character-level convolutional networks for text classification
Zhang, X., Zhao, J. J., and LeCun, Y · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L · 2016
Earlier work this paper cites.
Membership inference attacks against machine learning models
Shokri, R., Stronati, M., Song, C., and Shmatikov, V · 2017
Earlier work this paper cites.
Privacy risk in machine learning: Analyzing the connection to overfitting
Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S · 2018
Earlier work this paper cites.
The secret sharer: Evaluating and testing unintended memorization in neural networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., and Song, D · 2019
Earlier work this paper cites.
LOGAN: Membership inference attacks against generative models
Hayes, J., Melis, L., Danezis, G., and De Cristofaro, E · 2019
Earlier work this paper cites.
Monte Carlo and reconstruction membership inference attacks against generative models
Hilprecht, B., Härterich, M., and Bernau, D · 2019
Earlier work this paper cites.
Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning
Nasr, M., Shokri, R., and Houmansadr, A · 2019
Earlier work this paper cites.
Sentence-BERT: Sentence embeddings using siamese BERT-networks
Reimers, N. and Gurevych, I · 2019
Earlier work this paper cites.
White-box vs black-box: Bayes optimal strategies for membership inference
Sablayrolles, A., Douze, M., Schmid, C., Ollivier, Y., and Jégou, H · 2019
Earlier work this paper cites.
Auditing data provenance in text-generation models
Song, C. and Shmatikov, V · 2019
Earlier work this paper cites.
Assessing privacy and quality of synthetic health data
Yale, A., Dash, S., Dutta, R., Guyon, I., Pavao, A., and Bennett, K. P · 2019
Earlier work this paper cites.
GAN-leaks: A taxonomy of membership inference attacks against generative models
Chen, D., Yu, N., Zhang, Y., and Fritz, M · 2020
Earlier work this paper cites.
What neural networks memorize and why: Discovering the long tail via influence estimation
Feldman, V. and Zhang, C · 2020
Earlier work this paper cites.
Auditing differentially private machine learning: How private is private SGD?
Jagielski, M., Ullman, J., and Oprea, A · 2020
Earlier work this paper cites.
Stolen memories: Leveraging model memorization for calibrated white-box membership inference
Leino, K. and Fredrikson, M · 2020
Earlier work this paper cites.
Analyzing information leakage of updates to natural language models
Zanella-Béguelin, S., Wutschitz, L., Tople, S., Rühle, V., Paverd, A., Ohrimenko, O., Köpf, B., and Brockschmidt, M · 2020
Cited alongside, same era.
Extracting training data from large language models
Carlini, N., Tramèr, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al · 2021
Cited alongside, same era.
Label-only membership inference attacks
Choquette-Choo, C. A., Tramèr, F., Carlini, N., and Papernot, N · 2021
Cited alongside, same era.
Differentially private n-gram extraction
Kim, K., Gopi, S., Kulkarni, J., and Yekhanin, S · 2021
Cited alongside, same era.
On the importance of difficulty calibration in membership inference attacks
Watson, L., Guo, C., Cormode, G., and Sablayrolles, A · 2021
Cited alongside, same era.
Opacus: User-friendly differential privacy library in pytorch
Harnessing large-language models to generate private synthetic text, 2023
Kurakin, A., Ponomareva, N., Syed, U., MacDermed, L., and Terzis, A · 2023
Later among the works it cites.
MoPe: Model perturbation based privacy attacks on language models
Li, M., Wang, J., Wang, J. G., and Neel, S · 2023
Later among the works it cites.
Membership inference attacks against language models via neighbourhood comparison
Mattern, J., Mireshghallah, F., Jin, Z., Schölkopf, B., Sachan, M., and Berg-Kirkpatrick, T · 2023
Later among the works it cites.
Achilles’ heels: vulnerable record identification in synthetic data publishing
Meeus, M., Guepin, F., Creţu, A.-M., and de Montjoye, Y.-A · 2023
Later among the works it cites.
Scalable extraction of training data from (production) language models, 2023
Nasr, M., Carlini, N., Hayase, J., Jagielski, M., Cooper, A. F., Ippolito, D., Choquette-Choo, C. A., Wallace, E., Tramèr, F., and Lee, K · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yousefpour, A., Shilov, I., Sablayrolles, A., Testuggine, D., Prasad, K., Malek, M., Nguyen, J., Ghosh, S., Bharadwaj, A., Zhao, J., et al · 2021
Cited alongside, same era.
Membership inference attacks against synthetic health data
Zhang, Z., Yan, C., and Malin, B. A · 2021
Cited alongside, same era.
Membership inference attacks from first principles
Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tramèr, F · 2022
Cited alongside, same era.
The privacy onion effect: Memorization is relative
Carlini, N., Jagielski, M., Zhang, C., Papernot, N., Terzis, A., and Tramèr, F · 2022
Cited alongside, same era.
TAPAS: a toolbox for adversarial privacy auditing of synthetic data
Houssiau, F., Jordon, J., Cohen, S. N., Daniel, O., Elliott, A., Geddes, J., Mole, C., Rangel-Smith, C., and Szpruch, L · 2022
Cited alongside, same era.
LoRA: Low-rank adaptation of large language models
Hu, E. J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al · 2022
Cited alongside, same era.
An empirical study on the membership inference attack against tabular data synthesis models
Hyeong, J., Kim, J., Park, N., and Jajodia, S · 2022
Cited alongside, same era.
Later among the works it cites.
SoK: Let the privacy games begin! A unified treatment of data inference privacy in machine learning
Salem, A., Cherubin, G., Evans, D., Köpf, B., Paverd, A., Suri, A., Tople, S., and Zanella-Béguelin, S · 2023
Later among the works it cites.
Synthetic text generation with differential privacy: A simple and practical recipe
Yue, X., Inan, H., Li, X., Kumar, G., McAnallen, J., Shajari, H., Sun, H., Levitan, D., and Sim, R · 2023
Later among the works it cites.
Bayesian estimation of differential privacy
Zanella-Béguelin, S., Wutschitz, L., Tople, S., Salem, A., Rühle, V., Paverd, A., Naseri, M., Köpf, B., and Jones, D · 2023
Later among the works it cites.
Private prediction for large-scale synthetic text generation
Amin, K., Bie, A., Kong, W., Kurakin, A., Ponomareva, N., Syed, U., Terzis, A., and Vassilvitskii, S · 2024
Later among the works it cites.
Context-aware membership inference attacks against pre-trained large language models, 2024
Chang, H., Shamsabadi, A. S., Katevas, K., Haddadi, H., and Shokri, R · 2024
Later among the works it cites.
Investigating the effect of misalignment on membership privacy in the white-box setting
Cretu, A.-M., Jones, D., de Montjoye, Y.-A., and Tople, S · 2024
Later among the works it cites.
Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models
Jurafsky, D. and Martin, J. H · 2024
Later among the works it cites.
Copyright traps for large language models
Meeus, M., Shilov, I., Faysse, M., and de Montjoye, Y.-A · 2024
Later among the works it cites.
Detecting pretraining data from large language models
Shi, W., Ajith, A., Xia, M., Huang, Y., Liu, D., Blevins, T., Chen, D., and Zettlemoyer, L · 2024
Later among the works it cites.
Privacy-preserving in-context learning with differentially private few-shot generation
Tang, X., Shin, R., Inan, H. A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R · 2024
Later among the works it cites.
Proving membership in LLM pretraining data via data watermarks, 2024
Wei, J. T.-Z., Wang, R. Y., and Jia, R · 2024
Later among the works it cites.
Privacy-preserving in-context learning for large language models
Wu, T., Panda, A., Wang, J. T., and Mittal, P · 2024
Later among the works it cites.
Differentially private synthetic data via foundation model APIs 2: Text
Xie, C., Lin, Z., Backurs, A., Gopi, S., Yu, D., Inan, H. A., Nori, H., Jiang, H., Zhang, H., Lee, Y. T., Li, B., and Yekhanin, S · 2024
Later among the works it cites.
Low-cost high-power membership inference attacks
Zarifzadeh, S., Liu, P., and Shokri, R · 2024
Later among the works it cites.