The Tug of War Within: Mitigating the Fairness-Privacy Conflicts in Large Language Models

A definition of conditional mutual information for arbitrary ensembles

Aaron D Wyner. 1978 · 1978

Earlier work this paper cites.

Evaluation metrics for language models

Stanley F Chen, Douglas Beeferman, and Roni Rosenfeld. 1998 · 1998

Earlier work this paper cites.

Secure multi-party computation problems and their applications: a review and open problems

Wenliang Du and Mikhail J Atallah. 2001 · 2001

Earlier work this paper cites.

Mutual information theory for adaptive mixture models

Zheng Rong Yang and Mark Zwolinski. 2001 · 2001

Earlier work this paper cites.

Estimating mutual information

Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. 2004 · 2004

Earlier work this paper cites.

Privacy in deep learning: A survey

Original

Fatemehsadat Mireshghallah, Mohammadkazem Taram, Praneeth Vepakomma, Abhishek Singh, Ramesh Raskar, and Hadi Esmaeilzadeh. 2020 · 2004

Earlier work this paper cites.

Measuring statistical dependence with hilbert-schmidt norms

Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. 2005 · 2005

Earlier work this paper cites.

Differential privacy

Cynthia Dwork. 2006 · 2006

Earlier work this paper cites.

Calibrating noise to sensitivity in private data analysis

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006 · 2006

Earlier work this paper cites.

Learning word vectors for sentiment analysis

Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011 · 2011

Earlier work this paper cites.

Information theory

Robert B Ash. 2012 · 2012

Earlier work this paper cites.

Fairness through awareness

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012 · 2012

Earlier work this paper cites.

Striving for Simplicity: The All Convolutional Net

J Springenberg, Alexey Dosovitskiy, Thomas Brox, and M Riedmiller. 2015 · 2015

Earlier work this paper cites.

Character-level convolutional networks for text classification

Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015 · 2015

Earlier work this paper cites.

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks

Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2016 · 2016

Earlier work this paper cites.

Counterfactual fairness

Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017 · 2017

Earlier work this paper cites.

RACE: Large-scale ReAding comprehension dataset from examinations

Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017 · 2017

Earlier work this paper cites.

Membership inference attacks against machine learning models

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017 · 2017

Earlier work this paper cites.

Learning Important Features Through Propagating Activation Differences

Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017 · 2017

Earlier work this paper cites.

Axiomatic Attribution for Deep Networks

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017 · 2017

Earlier work this paper cites.

Can a suit of armor conduct electricity? a new dataset for open book question answering

Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018 · 2018

Earlier work this paper cites.

Fairness definitions explained

Sahil Verma and Julia Rubin. 2018 · 2018

Earlier work this paper cites.

Differential privacy has disparate impact on model accuracy

Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. 2019 · 2019

Earlier work this paper cites.

BoolQ: Exploring the surprising difficulty of natural yes/no questions

Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

On the compatibility of privacy and fairness

Rachel Cummings, Varun Gupta, Dhamma Kimpara, and Jamie Morgenstern. 2019 · 2019

Earlier work this paper cites.

BERT: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Designing and Interpreting Probes With Control Tasks

John Hewitt and Percy Liang. 2019 · 2019

Earlier work this paper cites.

Parameter-efficient transfer learning for nlp

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019 · 2019

Earlier work this paper cites.

Decoupled weight decay regularization

Ilya Loshchilov and Frank Hutter. 2019 · 2019

Earlier work this paper cites.

Are sixteen heads really better than one?

Paul Michel, Omer Levy, and Graham Neubig. 2019 · 2019

Earlier work this paper cites.

On variational bounds of mutual information

Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. 2019 · 2019

Earlier work this paper cites.

HellaSwag: Can a machine really finish your sentence?

Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019 · 2019

Earlier work this paper cites.

Privacy and security issues in deep learning: A survey

Ximeng Liu, Lehui Xie, Yaopeng Wang, Jian Zou, Jinbo Xiong, Zuobin Ying, and Athanasios V Vasilakos. 2020 · 2020

Earlier work this paper cites.

Differentially private representation for NLP: Formal guarantee and an empirical study on privacy and fairness

Lingjuan Lyu, Xuanli He, and Yitong Li. 2020 · 2020

Earlier work this paper cites.

The hsic bottleneck: Deep learning without back-propagation

Wan-Duo Kurt Ma, JP Lewis, and W Bastiaan Kleijn. 2020 · 2020

Earlier work this paper cites.

A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

Erico Tjoa and Cuntai Guan. 2020 · 2020

Cited alongside, same era.

Asking and answering questions to evaluate the factual consistency of summaries

Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020 · 2020

Cited alongside, same era.

Trade-offs between fairness and privacy in machine learning

Sushant Agarwal. 2021 · 2021

Cited alongside, same era.

Summeval: Re-evaluating summarization evaluation

Alexander R Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021 · 2021

Cited alongside, same era.

Transformer feed-forward layers are key-value memories

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021 · 2021

Cited alongside, same era.

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2021

Stanford alpaca: An instruction-following llama model

Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023 · 2023

Later among the works it cites.

Function Vectors in Large Language Models

Original

Eric Todd, Millicent L Li, Arnab Sen Sharma, Aaron Mueller, Byron C Wallace, and David Bau. 2023 · 2023

Later among the works it cites.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 · 2023

Later among the works it cites.

Label words are anchors: An information flow perspective for understanding in-context learning

Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, and Xu Sun. 2023a · 2023

Later among the works it cites.

Large language models illuminate a progressive pathway to artificial healthcare assistant: A review