Fetching the paper…
Reading the bibliography…
As the deployment of pre-trained language models (PLMs) expands, pressing security concerns have arisen regarding the potential for malicious extraction of training data, posing a threat to data privacy.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, et al. 2020 · 1901
Earlier work this paper cites.
Membership inference attacks from first principles
Nicholas Carlini, Steve Chien, Milad Nasr, et al. 2022 · 1914
Earlier work this paper cites.
Label-only membership inference attacks
Christopher A. Choquette-Choo, Florian Tramer, Nicholas Carlini, et al. 2021 · 1974
Earlier work this paper cites.
A learning algorithm for boltzmann machines
David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. 1985 · 1985
Earlier work this paper cites.
A neural probabilistic language model
Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. 2000 · 2000
Earlier work this paper cites.
Scaling laws for neural language models
Jared Kaplan, Sam McCandlish, Tom Henighan, et al. 2020 · 2001
Earlier work this paper cites.
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
What we talk about when we talk about context
Paul Dourish. 2004 · 2004
Earlier work this paper cites.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, et al. 2006 · 2006
Earlier work this paper cites.
Quantifying membership inference vulnerability via generalization gap and other model metrics
Jason W Bentley, Daniel Gibney, Gary Hoppenworth, et al. 2020 · 2009
Earlier work this paper cites.
Privacy in Context
Helen Nissenbaum. 2009 · 2009
Earlier work this paper cites.
Training production language models without memorizing user data
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, et al. 2020 · 2009
Earlier work this paper cites.
Comparison and evaluation of code clone detection techniques and tools: A qualitative approach
Chanchal K Roy, James R Cordy, and Rainer Koschke. 2009 · 2009
Earlier work this paper cites.
Scaling laws for autoregressive generative modeling
Tom Henighan, Jared Kaplan, Mor Katz, et al. 2020 · 2010
Earlier work this paper cites.
Recurrent neural network based language model
Tomas Mikolov, Martin Karafiát, Lukas Burget, et al. 2010 · 2010
Earlier work this paper cites.
An evaluation framework for plagiarism detection
Martin Potthast, Benno Stein, Alberto Barrón-Cedeño, and Paolo Rosso. 2010 · 2010
Earlier work this paper cites.
Model inversion attacks that exploit confidence information and basic countermeasures
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015 · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. 2015 · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, et al. 2016 · 2016
Earlier work this paper cites.
A diversity-promoting objective function for neural conversation models
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016 · 2016
Earlier work this paper cites.
ReCon: Revealing and controlling PII leaks in mobile network traffic
Jingjing Ren, Ashwin Rao, Martina Lindorfer, et al. 2016 · 2016
Earlier work this paper cites.
Deep variational information bottleneck
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, et al. 2017 · 2017
Earlier work this paper cites.
Obfuscation-resilient privacy leak detection for mobile apps through differential analysis
Andrea Continella, Yanick Fratantonio, Martina Lindorfer, et al. 2017 · 2017
Earlier work this paper cites.
Membership inference attacks against machine learning models
Reza Shokri, Marco Stronati, Congzheng Song, et al. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. 2017 · 2017
Earlier work this paper cites.
Hierarchical neural story generation
Angela Fan, Mike Lewis, and Yann Dauphin. 2018 · 2018
Earlier work this paper cites.
Ethical challenges in Data-Driven dialogue systems
Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, et al. 2018 · 2018
Earlier work this paper cites.
Humans forget, machines remember: Artificial intelligence and the right to be forgotten
Tiffany Li, Eduard Fosch Villaronga, and Peter Kieseberg. 2018 · 2018
Earlier work this paper cites.
Learning differentially private recurrent language models
H. Brendan McMahan, Daniel Ramage, Kunal Talwar, et al. 2018 · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, et al. 2018 · 2018
Earlier work this paper cites.
Privacy risk in machine learning: Analyzing the connection to overfitting
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, et al. 2018 · 2018
Earlier work this paper cites.
The adverse effects of code duplication in machine learning models of code
Miltiadis Allamanis. 2019 · 2019
Earlier work this paper cites.
The secret sharer: Evaluating and testing unintended memorization in neural networks
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, et al. 2019 · 2019
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
Making AI forget you: data deletion in machine learning
Antonio A Ginart, Melody Y Guan, Gregory Valiant, et al. 2019 · 2019
Earlier work this paper cites.
Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks
Jiasen Lu, Dhruv Batra, Devi Parikh, et al. 2019 · 2019
Earlier work this paper cites.
Exploiting unintended feature leakage in collaborative learning
Luca Melis, Congzheng Song, Emiliano De Cristofaro, et al. 2019 · 2019
Earlier work this paper cites.
Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning
Milad Nasr, Reza Shokri, and Amir Houmansadr. 2019 · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, et al. 2019 · 2019
Earlier work this paper cites.
Auditing data provenance in Text-Generation models
Congzheng Song and Vitaly Shmatikov. 2019 · 2019
Earlier work this paper cites.
LEGAL-BERT: The muppets straight out of law school
Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020 · 2020
Earlier work this paper cites.
What neural networks memorize and why: discovering the long tail via influence estimation
Vitaly Feldman and Chiyuan Zhang. 2020 · 2020
Earlier work this paper cites.
The pile: An 800GB dataset of diverse text for language modeling
Leo Gao, Stella Biderman, Sid Black, et al. 2020 · 2020
Earlier work this paper cites.
Formalizing data deletion in the context of the right to be forgotten
Sanjam Garg, Shafi Goldwasser, and Prashant Nalini Vasudevan. 2020 · 2020
Cited alongside, same era.
Membership inference attacks on sequence-to-sequence models: Is my data in your machine translation system?
Sorami Hisamoto, Matt Post, and Kevin Duh. 2020 · 2020
Cited alongside, same era.
The curious case of neural text degeneration
Ari Holtzman, Jan Buys, Li Du, et al. 2020 · 2020
Cited alongside, same era.
Auditing differentially private machine learning: how private is private SGD?
Matthew Jagielski, Jonathan Ullman, and Alina Oprea. 2020 · 2020
Cited alongside, same era.
Weight poisoning attacks on pretrained models
Keita Kurita, Paul Michel, and Graham Neubig. 2020 · 2020
Cited alongside, same era.
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Membership inference attacks on machine learning: A survey
Hongsheng Hu, Zoran Salcic, Lichao Sun, et al. 2022 · 2022
Later among the works it cites.
Preventing verbatim memorization in language models gives a false sense of privacy
Daphne Ippolito, Florian Tramèr, Milad Nasr, et al. 2022 · 2022
Later among the works it cites.
Semantic shift stability: Efficient way to detect performance degradation of word embeddings and pre-trained language models
Shotaro Ishihara, Hiromu Takahashi, and Hono Shirai. 2022 · 2022
Later among the works it cites.
Deduplicating training data mitigates privacy risks in language models
Nikhil Kandpal, Eric Wallace, and Colin Raffel. 2022 · 2022
Later among the works it cites.
Deduplicating training data makes language models better
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2022 · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, et al. 2020 · 2020
Cited alongside, same era.
Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training
Gen Li, Nan Duan, Yuejian Fang, Ming Gong, and Daxin Jiang. 2020 · 2020
Cited alongside, same era.
KART: Parameterization of privacy leakage scenarios from pre-trained language models
Yuta Nakamura, Shouhei Hanaoka, Yukihiro Nomura, et al. 2020 · 2020
Cited alongside, same era.
BioMegatron: Larger biomedical domain language model
Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, and Raghav Mani. 2020 · 2020
Cited alongside, same era.
Information leakage in embedding models
Congzheng Song and Ananth Raghunathan. 2020 · 2020
Cited alongside, same era.
The secret revealer: Generative model-inversion attacks against deep neural networks
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, et al. 2020 · 2020
Cited alongside, same era.
On the dangers of stochastic parrots: Can language models be too big?
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, et al. 2021 · 2021
Cited alongside, same era.
SafeText: A benchmark for exploring physical safety in language models
Sharon Levy, Emily Allaway, Melanie Subbiah, Lydia Chilton, Desmond Patton, Kathleen McKeown, and William Yang Wang. 2022 · 2022
Later among the works it cites.
Large language models can be strong differentially private learners
Xuechen Li, Florian Tramer, Percy Liang, et al. 2022 · 2022
Later among the works it cites.
BioGPT: generative pre-trained transformer for biomedical text generation and mining
Renqian Luo, Liai Sun, Yingce Xia, et al. 2022 · 2022
Later among the works it cites.
Sentence-level privacy for document embeddings
Casey Meehan, Khalil Mrini, and Kamalika Chaudhuri. 2022 · 2022
Later among the works it cites.
Mitigating covertly unsafe text within natural language systems
Alex Mei, Anisha Kabir, Sharon Levy, Melanie Subbiah, Emily Allaway, John Judge, Desmond Patton, Bruce Bimber, Kathleen McKeown, and William Yang Wang. 2022 · 2022
Later among the works it cites.
Quantifying privacy risks of masked language models using membership inference attacks
Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022a · 2022
Later among the works it cites.
An empirical analysis of memorization in fine-tuned autoregressive language models
Fatemehsadat Mireshghallah, Archit Uniyal, Tianhao Wang, David Evans, and Taylor Berg-Kirkpatrick. 2022b · 2022
Later among the works it cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang, et al. 2022 · 2022
Later among the works it cites.
Canary extraction in natural language understanding models
Rahil Parikh, Christophe Dupuy, and Rahul Gupta. 2022 · 2022
Later among the works it cites.
Red teaming language models with language models
Ethan Perez, Saffron Huang, Francis Song, et al. 2022 · 2022
Later among the works it cites.
Large language models encode clinical knowledge
Karan Singhal, Shekoofeh Azizi, Tao Tu, et al. 2022 · 2022
Later among the works it cites.
Shaden Smith, Mostofa Patwary, Brandon Norick, et al. 2022 · 2022
Later among the works it cites.
A contrastive framework for neural text generation
Yixuan Su, Tian Lan, Yan Wang, et al. 2022 · 2022
Later among the works it cites.
Galactica: A large language model for science
Ross Taylor, Marcin Kardas, Guillem Cucurull, et al. 2022 · 2022
Later among the works it cites.
Memorization without overfitting: Analyzing the training dynamics of large language models
Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, et al. 2022 · 2022
Later among the works it cites.
Considerations for differentially private learning with Large-Scale public pretraining
Florian Tramèr, Gautam Kamath, and Nicholas Carlini. 2022 · 2022
Later among the works it cites.
Downstream task performance of BERT models pre-trained using automatically de-identified clinical data
Thomas Vakili, Anastasios Lamproudis, Aron Henriksson, and Hercules Dalianis. 2022 · 2022
Later among the works it cites.
Will we run out of data? an analysis of the limits of scaling datasets in machine learning
Pablo Villalobos, Jaime Sevilla, Lennart Heim, et al. 2022 · 2022
Later among the works it cites.
Taxonomy of risks posed by language models
Laura Weidinger, Jonathan Uesato, Maribeth Rauh, et al. 2022 · 2022
Later among the works it cites.
A large language model for electronic health records
Xi Yang, Aokun Chen, Nima PourNejatian, et al. 2022 · 2022
Later among the works it cites.
Privacy-preserving models for legal natural language processing
Ying Yin and Ivan Habernal. 2022 · 2022
Later among the works it cites.
Differentially private fine-tuning of language models
Da Yu, Saurabh Naik, Arturs Backurs, et al. 2022 · 2022
Later among the works it cites.
Text revealer: Private text reconstruction via model inversion attacks against transformers
Ruisi Zhang, Seira Hidano, and Farinaz Koushanfar. 2022 · 2022
Later among the works it cites.
MusicLM: Generating music from text
Andrea Agostinelli, Timo I Denk, Zalán Borsos, et al. 2023 · 2023
Closest in time.
Emergent and predictable memorization in large language models
Stella Biderman, Usvsn Sai Prashanth, Lintang Sutawika, et al. 2023 · 2023
Closest in time.
Speak, memory: An archaeology of books known to ChatGPT/GPT-4
Kent K Chang, Mackenzie Cramer, Sandeep Soni, et al. 2023 · 2023
Closest in time.
Exploring the limits of differentially private deep learning with group-wise clipping
Jiyan He, Xuechen Li, Da Yu, et al. 2023 · 2023
Closest in time.
A VAE for transformers with nonparametric variational information bottleneck
James Henderson and Fabio James Fehr. 2023 · 2023
Closest in time.
Measuring forgetting of memorized training examples
Matthew Jagielski, Om Thakkar, Florian Tramèr, et al. 2023 · 2023
Closest in time.
Do language models plagiarize?
Jooyoung Lee, Thai Le, Jinghui Chen, et al. 2023 · 2023
Closest in time.
Analyzing leakage of personally identifiable information in language models
Nils Lukas, Ahmed Salem, Robert Sim, et al. 2023 · 2023
Closest in time.
Auditing large language models: a three-layered approach
Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi. 2023 · 2023
Closest in time.
OpenAI. 2023 · 2023
Closest in time.
Differentially private In-Context learning
Ashwinee Panda, Tong Wu, Jiachen T Wang, et al. 2023 · 2023
Closest in time.
BloombergGPT: A large language model for finance
Shijie Wu, Ozan Irsoy, Steven Lu, et al. 2023 · 2023
Closest in time.
Harnessing the power of LLMs in practice: A survey on ChatGPT and beyond
Jingfeng Yang, Hongye Jin, Ruixiang Tang, et al. 2023 · 2023
Closest in time.
Label-only model inversion attacks: Attack with the least information
Tianqing Zhu, Dayong Ye, Shuai Zhou, et al. 2023 · 2023
Closest in time.
Are large pre-trained language models leaking your personal information?
Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. 2022 · 2047
Closest in time.