PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration

Bert rediscovers the classical nlp pipeline

Original

Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019 · 1905

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Mechanism design via differential privacy

Frank McSherry and Kunal Talwar. 2007 · 2007

Earlier work this paper cites.

Privacy and the Internet Your Expectations and Rights Under the Law

Jasper M C. 2009 · 2009

Earlier work this paper cites.

A differentially private text perturbation method using a regularized mahalanobis metric

Original

Zekun Xu, Abhinav Aggarwal, Oluwaseyi Feyisetan, and Nathanael Teissier. 2020 · 2010

Earlier work this paper cites.

Broadening the scope of differential privacy using metrics

Konstantinos Chatzikokolakis, Miguel E. Andrés, Nicolás Emilio Bordenabe, and Catuscia Palamidessi. 2013 · 2013

Earlier work this paper cites.

Local privacy and statistical minimax rates

John C. Duchi, Michael I. Jordan, and Martin J. Wainwright. 2013 · 2013

Earlier work this paper cites.

Privacy notices versus informational self-determination: Minding the gap

B Van Alsenoy, E Kosta, and J Dumortier. 2014 · 2014

Earlier work this paper cites.

Understanding intermediate layers using linear classifier probes

Original

Guillaume Alain and Yoshua Bengio. 2016 · 2016

Earlier work this paper cites.

Calibrating noise to sensitivity in private data analysis

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. 2016 · 2016

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Invited paper: Local differential privacy on metric spaces: Optimizing the trade-off with utility

Mário S. Alvim, Konstantinos Chatzikokolakis, Catuscia Palamidessi, and Anna Pazii. 2018 · 2018

Earlier work this paper cites.

BERT: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Leveraging hierarchical representations for preserving privacy and utility in text

Oluwaseyi Feyisetan, Tom Diethe, and Thomas Drake. 2019 · 2019

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019

Earlier work this paper cites.

Privacy-and utility-preserving textual analysis via calibrated multivariate perturbations

Oluwaseyi Feyisetan, Borja Balle, Thomas Drake, and Tom Diethe. 2020 · 2020

Earlier work this paper cites.

ER-AE: differentially private text generation for authorship anonymization

Haohan Bo, Steven H. H. Ding, Benjamin C. M. Fung, and Farkhund Iqbal. 2021 · 2021

Earlier work this paper cites.

Extracting training data from large language models

Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. 2021 · 2021

Cited alongside, same era.

Probing classifiers: Promises, shortcomings, and advances

Yonatan Belinkov. 2022 · 2022

Cited alongside, same era.

You don’t know my favorite color: Preventing dialogue representations from revealing speakers’ private personas

Haoran Li, Yangqiu Song, and Lixin Fan. 2022 · 2022

Cited alongside, same era.

Differentially private language models for secure data sharing

Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, and Mrinmaya Sachan. 2022a · 2022

Cited alongside, same era.

The limits of word level differential privacy

Justus Mattern, Benjamin Weggenmann, and Florian Kerschbaum. 2022b · 2022

Cited alongside, same era.

Ddxplus: A new dataset for automatic medical diagnosis

Activation addition: Steering language models without optimization

Original

Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. 2023 · 2023

Later among the works it cites.

Locally differentially private document generation using zero shot prompting

Saiteja Utpala, Sara Hooker, and Pin-Yu Chen. 2023b · 2023

Later among the works it cites.

Bloomberggpt: A large language model for finance

Original

Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023 · 2023

Later among the works it cites.

Judging llm-as-a-judge with mt-bench and chatbot arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023a · 2023

Later among the works it cites.

Primer: Fast private transformer inference on encrypted data

Mengxin Zheng, Qian Lou, and Lei Jiang. 2023b · 2023