Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation

Radford, A., Narasimhan, K., Salimans, T., Sutskever, I. & Others Improving language understanding by generative pre-training. (OpenAI,2018)

2018

Earlier work this paper cites.

Devlin, J., Chang, M., Lee, K. & Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv Preprint ArXiv:1810.04805

Original

2018

Earlier work this paper cites.

Zhang, B., Lemoine, B. & Mitchell, M. Mitigating unwanted biases with adversarial learning. Proceedings Of The 2018 AAAI/ACM Conference On AI, Ethics, And Society

2018

Earlier work this paper cites.

Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B. & Others The malicious use of artificial intelligence: Forecasting, prevention, and mitigation. ArXiv Preprint ArXiv:1802.07228

Original

2018

Earlier work this paper cites.

Sheng, E., Chang, K., Natarajan, P. & Peng, N. The woman worked as a babysitter: On biases in language generation. ArXiv Preprint ArXiv:1909.01326

Original

2019

Earlier work this paper cites.

Zmigrod, R., Mielke, S., Wallach, H. & Cotterell, R. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. ArXiv Preprint ArXiv:1906.04571

Original

2019

Earlier work this paper cites.

Qian, Y., Muaz, U., Zhang, B. & Hyun, J. Reducing gender bias in word-level language models with a gender-equalizing loss function. ArXiv Preprint ArXiv:1905.12801

Original

2019

Earlier work this paper cites.

Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K. & Wang, W. Mitigating gender bias in natural language processing: Literature review. ArXiv Preprint ArXiv:1906.08976

Original

2019

Earlier work this paper cites.

Brown, T. & Others Language models are few-shot learners. Advances In Neural Information Processing Systems

2020

Earlier work this paper cites.

Gehman, S., Gururangan, S., Sap, M., Choi, Y. & Smith, N. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. ArXiv Preprint ArXiv:2009.11462

Original

2020

Earlier work this paper cites.

Nadeem, M., Bethke, A. & Reddy, S. StereoSet: Measuring stereotypical bias in pretrained language models. ArXiv Preprint ArXiv:2004.09456

Original

2020

Earlier work this paper cites.

Kennedy, B., Jin, X., Davani, A., Dehghani, M. & Ren, X. Contextualizing hate speech classifiers with post-hoc explanation. ArXiv Preprint ArXiv:2005.02439

Original

2020

Earlier work this paper cites.

Wang, B., Xu, C., Wang, S., Gan, Z., Cheng, Y., Gao, J., Awadallah, A. & Li, B. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. ArXiv Preprint ArXiv:2111.02840

Original

2021

Earlier work this paper cites.

Bender, E., Gebru, T., McMillan-Major, A. & Shmitchell, S. On the dangers of stochastic parrots: Can language models be too big?. Proceedings Of The 2021 ACM Conference On Fairness, Accountability, And Transparency

2021

Earlier work this paper cites.

Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J. & Others Ethical and social risks of harm from language models. ArXiv Preprint ArXiv:2112.04359

Original

2021

Earlier work this paper cites.

Abid, A., Farooqi, M. & Zou, J. Persistent anti-muslim bias in large language models. Proceedings Of The 2021 AAAI/ACM Conference On AI, Ethics, And Society

2021

Earlier work this paper cites.

Dhamala, J., Sun, T., Kumar, V., Krishna, S., Pruksachatkun, Y., Chang, K. & Gupta, R. Bold: Dataset and metrics for measuring biases in open-ended language generation. Proceedings Of The 2021 ACM Conference On Fairness, Accountability, And Transparency

2021

Cited alongside, same era.

Schick, T., Udupa, S. & Schütze, H. Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in nlp. Trans. Assoc. Comput. Linguist

2021

Cited alongside, same era.

Srivastava, A., Rastogi, A., Rao, A., Shoeb, A., Abid, A., Fisch, A., Brown, A., Santoro, A., Gupta, A., Garriga-Alonso, A. & Others Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. ArXiv Preprint ArXiv:2206.04615

Original

2022

Cited alongside, same era.

Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D. & Kamar, E. ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection. Proceedings Of The 60th Annual Meeting Of The Association For Computational Linguistics

2022

Shen, X., Chen, Z., Backes, M., Shen, Y. & Zhang, Y. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. ArXiv Preprint ArXiv:2308.03825

Original

2023

Later among the works it cites.

Gupta, V., Venkit, P., Laurençon, H., Wilson, S. & Passonneau, R. Calm: A multi-task benchmark for comprehensive assessment of language model bias. ArXiv Preprint ArXiv:2308.12539

Original

2023

Later among the works it cites.

Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y., Li, Y., Lundberg, S. & Others Sparks of artificial general intelligence: Early experiments with gpt-4. ArXiv Preprint ArXiv:2303.12712

Original

2023

Later among the works it cites.

Navigli, R., Conia, S. & Ross, B. Biases in large language models: origins, inventory, and discussion. ACM Journal Of Data And Information Quality

2023

Later among the works it cites.