Built on

C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in Proceedings of the 3rd innovations in theoretical computer science conference , 2012, pp. 214–226
2012
Earlier work this paper cites.
M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” Advances in neural information processing systems , vol. 29, 2016
2016
Earlier work this paper cites.
M. J. Kusner, J. Loftus, C. Russell, and R. Silva, “Counterfactual fairness,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
M. B. Zafar, I. Valera, M. Gomez Rodriguez, and K. P. Gummadi, “Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment,” in Proceedings of the 26th international conference on world wide web , 2017, pp. 1171–1180
2017
Earlier work this paper cites.
J. Dressel and H. Farid, “The accuracy, fairness, and limits of predicting recidivism,” Science advances , vol. 4, no. 1, p. eaao5580, 2018
2018
Earlier work this paper cites.
E. Krasanakis, E. Spyromitros-Xioufis, S. Papadopoulos, and Y. Kompatsiaris, “Adaptive sensitive reweighting to mitigate bias in fairness-aware classification,” in Proceedings of the 2018 world wide web conference , 2018, pp. 853–862
2018
Earlier work this paper cites.
M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and K. P. Gummadi, “Fairness constraints: A flexible approach for fair classification,” The Journal of Machine Learning Research , vol. 20, no. 1, pp. 2737–2778, 2019
2019
Earlier work this paper cites.
S. Caton and C. Haas, “Fairness in machine learning: A survey,” arXiv preprint arXiv:2010.04053 , 2020
Original
2020
Earlier work this paper cites.
N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Computing Surveys (CSUR) , vol. 54, no. 6, pp. 1–35, 2021
2021
Earlier work this paper cites.

Similar

Then

J. Zhou, H. Müller, A. Holzinger, and F. Chen, “Ethical chatgpt: Concerns, challenges, and commandments,” 2023
2023
Closest in time.
Y. Liu, T. Han, S. Ma, J. Zhang, Y. Yang, J. Tian, H. He, A. Li, M. He, Z. Liu et al. , “Summary of chatgpt/gpt-4 research and perspective towards the future of large language models,” arXiv preprint arXiv:2304.01852 , 2023
Original
2023
Closest in time.
W. Hariri, “Unlocking the potential of chatgpt: A comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing,” arXiv preprint arXiv:2304.02017 , 2023
Original
2023
Closest in time.
H. Nori, N. King, S. M. McKinney, D. Carignan, and E. Horvitz, “Capabilities of gpt-4 on medical challenge problems,” arXiv preprint arXiv:2303.13375 , 2023
Original
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2023

Fairness of ChatGPT

Li, Yunqi, Zhang, Lanjing, Zhang, Yongfeng

Understand

Understanding and addressing unfairness in LLMs are crucial for responsible AI deployment.

However, there is a limited number of quantitative analyses and in-depth studies regarding fairness evaluations in LLMs, especially when applying LLMs to high-stakes fields.
This work aims to fill this gap by providing a systematic evaluation of the effectiveness and fairness of LLMs using ChatGPT as a study case.
We focus on assessing ChatGPT's performance in high-takes fields including education, criminology, finance and healthcare.

Built on

C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in Proceedings of the 3rd innovations in theoretical computer science conference , 2012, pp. 214–226
2012
Earlier work this paper cites.
M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” Advances in neural information processing systems , vol. 29, 2016
2016
Earlier work this paper cites.
M. J. Kusner, J. Loftus, C. Russell, and R. Silva, “Counterfactual fairness,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
M. B. Zafar, I. Valera, M. Gomez Rodriguez, and K. P. Gummadi, “Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment,” in Proceedings of the 26th international conference on world wide web , 2017, pp. 1171–1180
2017
Earlier work this paper cites.
J. Dressel and H. Farid, “The accuracy, fairness, and limits of predicting recidivism,” Science advances , vol. 4, no. 1, p. eaao5580, 2018
2018
Earlier work this paper cites.
E. Krasanakis, E. Spyromitros-Xioufis, S. Papadopoulos, and Y. Kompatsiaris, “Adaptive sensitive reweighting to mitigate bias in fairness-aware classification,” in Proceedings of the 2018 world wide web conference , 2018, pp. 853–862
2018
Earlier work this paper cites.
M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and K. P. Gummadi, “Fairness constraints: A flexible approach for fair classification,” The Journal of Machine Learning Research , vol. 20, no. 1, pp. 2737–2778, 2019
2019
Earlier work this paper cites.
S. Caton and C. Haas, “Fairness in machine learning: A survey,” arXiv preprint arXiv:2010.04053 , 2020
Original
2020
Earlier work this paper cites.
N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Computing Surveys (CSUR) , vol. 54, no. 6, pp. 1–35, 2021
2021
Earlier work this paper cites.

Similar

Then

J. Zhou, H. Müller, A. Holzinger, and F. Chen, “Ethical chatgpt: Concerns, challenges, and commandments,” 2023
2023
Closest in time.
Y. Liu, T. Han, S. Ma, J. Zhang, Y. Yang, J. Tian, H. He, A. Li, M. He, Z. Liu et al. , “Summary of chatgpt/gpt-4 research and perspective towards the future of large language models,” arXiv preprint arXiv:2304.01852 , 2023
Original
2023
Closest in time.
W. Hariri, “Unlocking the potential of chatgpt: A comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing,” arXiv preprint arXiv:2304.02017 , 2023
Original
2023
Closest in time.
H. Nori, N. King, S. M. McKinney, D. Carignan, and E. Horvitz, “Capabilities of gpt-4 on medical challenge problems,” arXiv preprint arXiv:2303.13375 , 2023
Original
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

T. Le Quy, A. Roy, V. Iosifidis, W. Zhang, and E. Ntoutsi, “A survey on datasets for fairness-aware machine learning,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , vol. 12, no. 3, p. e1452, 2022

2022

Cited alongside, same era.

OpenAI, “Gpt-4 technical report,” arXiv , 2023

2023

Cited alongside, same era.

Y. Ge, W. Hua, K. Mei, jianchao ji, J. Tan, S. Xu, Z. Li, and Y. Zhang, “OpenAGI: When LLM meets domain experts,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023

2023

Cited alongside, same era.

Y. Li, H. Chen, S. Xu, Y. Ge, J. Tan, S. Liu, and Y. Zhang, “Fairness in recommendation: Foundations, methods, and applications,” ACM Transactions on Intelligent Systems and Technology , vol. 14, no. 5, pp. 1–48, 2023

2023

Cited alongside, same era.

T. Y. Zhuo, Y. Huang, C. Chen, and Z. Xing, “Exploring ai ethics of chatgpt: A diagnostic analysis,” arXiv preprint arXiv:2301.12867 , 2023

Original

2023

Cited alongside, same era.

“Heart disease data set,” https://archive.ics.uci.edu/ml/datasets/heart+disease

Cited in the paper.

G. Lin and Y. Zhang, “Sparks of artificial general recommender (agr): Experiments with chatgpt,” Algorithms , vol. 16, no. 9, p. 432, 2023

2023

Closest in time.

H. Sun, Z. Zhang, J. Deng, J. Cheng, and M. Huang, “Safety assessment of chinese large language models,” arXiv preprint arXiv:2304.10436 , 2023

Original

2023

Closest in time.

T. Le Quy, T. H. Nguyen, G. Friege, and E. Ntoutsi, “Evaluation of group fairness measures in student performance prediction problems,” in Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2022, Grenoble, France, September 19–23, 2022, Proceedings, Part I . Springer, 2023, pp. 119–136

2023

Closest in time.

D. Chen and L. Zhang, “Monotonicity for ai ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and finance,” arXiv preprint arXiv:2301.07060 , 2023

Original

2023

Closest in time.

W. Hua, Y. Ge, S. Xu, J. Ji, and Y. Zhang, “Up5: Unbiased foundation model for fairness-aware recommendation,” EACL , 2024

2024

Closest in time.