Fetching the paper…
Reading the bibliography…
Pre-trained large language models, such as GPT\nobreakdash-2 and BERT, are often fine-tuned to achieve state-of-the-art performance on a downstream task.
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith · 2006
Earlier work this paper cites.
10k most common passwords, 2011
Mark Burnett · 2011
Earlier work this paper cites.
Opinion 05/2014 on “Anonymisation Techniques”, 2014
Art. 29 WP · 2014
Earlier work this paper cites.
Hillary clinton’s emails, 2015
Kaggle Competition · 2015
Earlier work this paper cites.
Deep dive: Eff’s new wordlists for random passphrases, 2016
Joseph Bonneau · 2016
Earlier work this paper cites.
Smart reply: Automated response suggestion for email
Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufman, Balint Miklos, Greg Corrado, Andrew Tomkins, Laszlo Lukacs, Marina Ganea, Peter Young, and Vivek Ramavajjala · 2016
Earlier work this paper cites.
Self-Es: The role of emails-to-self in personal information management
Horatiu Bota, Paul Bennett, Ahmed H. Awadallah, and Susan Dumais · 2017
Earlier work this paper cites.
Efficient natural language response suggestion for smart reply
Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun hsuan Sung, László Lukács, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil · 2017
Earlier work this paper cites.
Building smart replies for member messages
Jeff Pasternack, Nimesh Chakravarthi, Adam Leon, Nandeesh Rajashekar, Birjodh Tiwana, and Bing Zhao · 2017
Earlier work this paper cites.
Membership inference attacks against machine learning models
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Property inference attacks on fully connected neural networks using permutation invariant representations
Karan Ganju, Qi Wang, Wei Yang, Carl A. Gunter, and Nikita Borisov · 2018
Earlier work this paper cites.
Understanding membership inferences on well-generalized learning models
Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen · 2018
Earlier work this paper cites.
Towards demystifying membership inference attacks
Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei · 2018
Earlier work this paper cites.
Privacy risk in machine learning: Analyzing the connection to overfitting
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha · 2018
Earlier work this paper cites.
The secret sharer: Evaluating and testing unintended memorization in neural networks
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song · 2019
Earlier work this paper cites.
Gmail smart compose: Real-time assisted writing
Mia Chen, Zhifeng Chen, Timothy Sohn, Yonghui Wu, Benjamin Lee, Gagan Bansal, Yuan Cao, Shuyuan Zhang, Justin Lu, Jackie Tsay, Yinan Wang, and Andrew Dai · 2019
Earlier work this paper cites.
Diversifying reply suggestions using a matching-conditional variational autoencoder
Budhaditya Deb, Peter Bailey, and Milad Shokouhi · 2019
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Earlier work this paper cites.
LOGAN: Membership inference attacks against generative models
Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro · 2019
Earlier work this paper cites.
A repository of conversational datasets
Matthew Henderson, Paweł Budzianowski, Inigo Casanueva, Sam Coope, Daniela Gerz, Girish Kumar, Nikola Mrkšić, Georgios Spithourakis, Pei-Hao Su, Ivan Vulić, and Tsung-Hsien Wen · 2019
Cited alongside, same era.
Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning
Milad Nasr, Reza Shokri, and Amir Houmansadr · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Cited alongside, same era.
White-box vs black-box: Bayes optimal strategies for membership inference
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Hervé Jégou · 2019
Cited alongside, same era.
ML-Leaks: Model and data independent membership inference attacks and defenses on machine learning models
Ahmed Salem, Yang Zhang, Mathias Humbert, Mario Fritz, and Michael Backes · 2019
Does bert pretrained on clinical notes reveal sensitive data?
Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, and Byron C Wallace · 2021
Later among the works it cites.
Large language models can be strong differentially private learners, 2021
Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto · 2021
Later among the works it cites.
Membership inference attacks against nlp classification models
Virat Shejwalkar, Huseyin A. Inan, Amir Houmansadr, and Robert Sim · 2021
Later among the works it cites.
Bert2bert model, 2021
Patrick von Platen · 2021
Later among the works it cites.
GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model
Ben Wang and Aran Komatsuzaki · 2021
Later among the works it cites.
Leakage of dataset properties in Multi-Party machine learning
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Auditing data provenance in text-generation models
Congzheng Song and Vitaly Shmatikov · 2019
Cited alongside, same era.
OCC: A smart reply system for efficient in-app communications
Yue Weng, Huaixiu Zheng, Franziska Bell, and Gökhan Tür · 2019
Cited alongside, same era.
Language models are few-shot learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Cited alongside, same era.
Indian companies registration data [1857 - 2020], 2020
Kaggle Competition · 2020
Cited alongside, same era.
The curious case of neural text degeneration
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi · 2020
Cited alongside, same era.
Revisiting membership inference under realistic assumptions
Bargav Jayaraman, Lingxiao Wang, Katherine Knipmeyer, Quanquan Gu, and David Evans · 2020
Cited alongside, same era.
Stolen Memories: Leveraging model memorization for calibrated white-box membership inference
Klas Leino and Matt Fredrikson · 2020
Cited alongside, same era.
Wanrong Zhang, Shruti Tople, and Olga Ohrimenko · 2021
Later among the works it cites.
Quantifying memorization across neural language models
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang · 2022
Closest in time.
Introducing the world’s largest open multilingual language model: Bloom, 2022
HuggingFace · 2022
Closest in time.
Deduplicating training data mitigates privacy risks in language models
Nikhil Kandpal, Eric Wallace, and Colin Raffel · 2022
Closest in time.
Property inference from poisoning
Saeed Mahloujifar, Esha Ghosh, and Melissa Chase · 2022
Closest in time.
Introducing chatgpt
OpenAI · 2022
Closest in time.
Data Classification Standards
Susanne Windfeld Pedersen, Eva Dupont, Blanca Robledo Díaz, and Swetha Kundaram · 2022
Closest in time.
Truth serum: Poisoning machine learning models to reveal their secrets
Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini · 2022
Closest in time.
Emergent abilities of large language models
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus · 2022
Closest in time.
Opt: Open pre-trained transformer language models
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer · 2022
Closest in time.
Introducing Claude
Anthropic · 2023
Closest in time.
Analyzing leakage of personally identifiable information in language models
Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin · 2023
Closest in time.
Alpaca: A strong, replicable instruction-following model
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto · 2023
Closest in time.
Llama: Open and efficient foundation language models, 2023
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample · 2023
Closest in time.