Fetching the paper…
Reading the bibliography…
As Large Language Models (LLMs) become more pervasive across various users and scenarios, identifying potential issues when using these models becomes essential.
Bertscore: Evaluating text generation with bert
Zhang, T.; Kishore, V.; Wu, F.; Weinberger, K. Q.; and Artzi, Y. 2019 · 1904
Earlier work this paper cites.
Language Models as Knowledge Bases?
Petroni, F.; Rocktäschel, T.; Lewis, P.; Bakhtin, A.; Wu, Y.; Miller, A. H.; and Riedel, S. 2019 · 1909
Earlier work this paper cites.
How Can We Know What Language Models Know?
Jiang, Z.; Xu, F. F.; Araki, J.; and Neubig, G. 2020 · 1911
Earlier work this paper cites.
A Feasibility Study of Answer-Agnostic Question Generation for Education
Dugan, L.; Miltsakaki, E.; Upadhyay, S.; Ginsberg, E.; Gonzalez, H.; Choi, D.; Yuan, C.; and Callison-Burch, C. 2022 · 1926
Earlier work this paper cites.
ROUGE: A Package for Automatic Evaluation of Summaries
Lin, C.-Y. 2004 · 2004
Earlier work this paper cites.
Language (technology) is power: A critical survey of” bias” in nlp
Blodgett, S. L.; Barocas, S.; and Daumé III, H. 2020 · 2005
Earlier work this paper cites.
Inter-Coder Agreement for Computational Linguistics
Artstein, R.; and Poesio, M. 2008 · 2008
Earlier work this paper cites.
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S.; Gururangan, S.; Sap, M.; Choi, Y.; and Smith, N. A. 2020 · 2009
Earlier work this paper cites.
Computing Krippendorff’s Alpha-Reliability
Krippendorff, K. 2011 · 2011
Earlier work this paper cites.
Content analysis: An introduction to its methodology
Krippendorff, K. 2018 · 2018
Earlier work this paper cites.
BLEURT: Learning Robust Metrics for Text Generation
Sellam, T.; Das, D.; and Parikh, A. 2020 · 2020
Earlier work this paper cites.
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets
Blodgett, S. L.; Lopez, G.; Olteanu, A.; Sim, R.; and Wallach, H. 2021 · 2021
Earlier work this paper cites.
Mitigating political bias in language models through reinforced calibration
Liu, R.; Jia, C.; Wei, J.; Xu, G.; Wang, L.; and Vosoughi, S. 2021 · 2021
Earlier work this paper cites.
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N.; Kapania, S.; Highfill, H.; Akrong, D.; Paritosh, P.; and Aroyo, L. M. 2021 · 2021
Earlier work this paper cites.
Ask Me Anything: A simple strategy for prompting language models
Arora, S.; Narayan, A.; Chen, M. F.; Orr, L.; Guha, N.; Bhatia, K.; Chami, I.; Sala, F.; and Ré, C. 2022 · 2022
Earlier work this paper cites.
Towards WinoQueer: Developing a benchmark for anti-queer bias in large language models
Felkner, V. K.; Chang, H.-C. H.; Jang, E.; and May, J. 2022 · 2022
Earlier work this paper cites.
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
Ganguli, D.; Lovitt, L.; Kernion, J.; Askell, A.; Bai, Y.; Kadavath, S.; Mann, B.; Perez, E.; Schiefer, N.; Ndousse, K.; Jones, A.; Bowman, S.; Chen, A.; Conerly, T.; DasSarma, N.; Drain, D.; Elhage, N.; El-Showk, S.; Fort, S.; Hatfield-Dodds, Z.; Henighan, T.; Hernandez, D.; Hume, T.; Jacobson, J.; Johnston, S.; Kravec, S.; Olsson, C.; Ringer, S.; Tran-Johnson, E.; Amodei, D.; Brown, T.; Joseph, N.; McCandlish, S.; Olah, C.; Kaplan, J.; and Clark, J. 2022 · 2022
Earlier work this paper cites.
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Hartvigsen, T.; Gabriel, S.; Palangi, H.; Sap, M.; Ray, D.; and Kamar, E. 2022 · 2022
Earlier work this paper cites.
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S.; Hilton, J.; and Evans, O. 2022 · 2022
Earlier work this paper cites.
Red Teaming Language Models with Language Models
Perez, E.; Huang, S.; Song, F.; Cai, T.; Ring, R.; Aslanides, J.; Glaese, A.; McAleese, N.; and Irving, G. 2022 · 2022
Earlier work this paper cites.
Multiple-Choice Question Generation: Towards an Automated Assessment Framework
Raina, V.; and Gales, M. 2022 · 2022
Earlier work this paper cites.
Adaptive Testing and Debugging of NLP Models
Ribeiro, M. T.; and Lundberg, S. 2022 · 2022
Cited alongside, same era.
You reap what you sow: On the challenges of bias evaluation under multilingual settings
Talat, Z.; Névéol, A.; Biderman, S.; Clinciu, M.; Dey, M.; Longpre, S.; Luccioni, S.; Masoud, M.; Mitchell, M.; Radev, D.; Sharma, S.; Subramonian, A.; Tae, J.; Tan, S.; Tunuguntla, D.; and Van Der Wal, O. 2022 · 2022
Cited alongside, same era.
Intentional Biases in LLM Responses
Badyal, N.; Jacoby, D.; and Coady, Y. 2023 · 2023
Cited alongside, same era.
Hallucination detection: Robustly discerning reliable answers in large language models
Chen, Y.; Fu, Q.; Yuan, Y.; Wen, Z.; Fan, G.; Liu, D.; Zhang, D.; Li, Z.; and Xiao, Y. 2023 · 2023
Cited alongside, same era.
LLM-assisted content analysis: Using large language models to support deductive coding
Chew, R.; Bollenbacher, J.; Wenger, M.; Speer, J.; and Kim, A. 2023 · 2023
Cited alongside, same era.
Penedo, G.; Malartic, Q.; Hesslow, D.; Cojocaru, R.; Cappelli, A.; Alobeidli, H.; Pannier, B.; Almazrouei, E.; and Launay, J. 2023 · 2023
Later among the works it cites.
Supporting human-ai collaboration in auditing llms with llms
Rastogi, C.; Tulio Ribeiro, M.; King, N.; Nori, H.; and Amershi, S. 2023 · 2023
Later among the works it cites.
The political biases of chatgpt
Rozado, D. 2023 · 2023
Later among the works it cites.
The Self-Perception and Political Biases of ChatGPT
Rutinowski, J.; Franke, S.; Endendyk, J.; Dormuth, I.; and Pauly, M. 2023 · 2023
Later among the works it cites.
DelucionQA: Detecting Hallucinations in Domain-specific Question Answering
Sadat, M.; Zhou, Z.; Lange, L.; Araki, J.; Gundroo, A.; Wang, B.; Menon, R.; Parvez, M.; and Feng, Z. 2023 · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ghosh, S.; and Caliskan, A. 2023 · 2023
Cited alongside, same era.
Political Bias in Large Language Models
Gover, L. 2023 · 2023
Cited alongside, same era.
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers
Guo, Q.; Wang, R.; Guo, J.; Li, B.; Song, K.; Tan, X.; Liu, G.; Bian, J.; and Yang, Y. 2023 · 2023
Cited alongside, same era.
Hartmann, J.; Schwenzow, J.; and Witte, M. 2023 · 2023
Cited alongside, same era.
ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learning
Hasanbeig, H.; Sharma, H.; Betthauser, L.; Vieira Frujeri, F.; and Momennejad, I. 2023 · 2023
Cited alongside, same era.
ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model
Huang, H.; Zheng, O.; Wang, D.; Yin, J.; Wang, Z.; Ding, S.; Yin, H.; Xu, C.; Yang, R.; Zheng, Q.; et al. 2023 · 2023
Cited alongside, same era.
Survey of hallucination in natural language generation
Ji, Z.; Lee, N.; Frieske, R.; Yu, T.; Su, D.; Xu, Y.; Ishii, E.; Bang, Y. J.; Madotto, A.; and Fung, P. 2023 · 2023
Cited alongside, same era.
Shaping the Emerging Norms of Using Large Language Models in Social Computing Research
Shen, H.; Li, T.; Li, T. J.-J.; Park, J. S.; and Yang, D. 2023a · 2023
Later among the works it cites.
Large Language Models Help Humans Verify Truthfulness–Except When They Are Convincingly Wrong
SI, C.; GOYAL, N.; WU, S. T.; ZHAO, C.; FENG, S.; DAUMÉ III, H.; and BOYD-GRABER, J. 2023 · 2023
Later among the works it cites.
Evaluating the factual consistency of large language models through news summarization
Tam, D.; Mascarenhas, A.; Zhang, S.; Kwan, S.; Bansal, M.; and Raffel, C. 2023 · 2023
Later among the works it cites.
Thakur, V. 2023 · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; Bikel, D.; Blecher, L.; CantonFerrer, C.; Chen, M.; Cucurull, G.; Esiobu, D.; Fernandes, J.; Fu, J.; Fu, W.; Fuller, B.; Gao, C.; Goswami, V.; Goyal, N.; Hartshorn, A.; Hosseini, S.; Hou, R.; Inan, H.; Kardas, M.; Kerkez, V.; Khabsa, M.; Kloumann, I.; Korenev, A.; ; Koura, P. S.; Lachaux, M.-A.; Lavril, T.; Lee, J.; Liskovich, D.; Lu, Y.; Mao, Y.; Martinet, X.; Mihaylov, T.; Mishra, P.; Molybog, I.; Nie, Y.; Poulton, A.; Reizenstein, J.; Rungta, R.; Saladi, K.; Schelten, A.; Silva, R.; Smith, E. M.; Subramanian, R.; Tan, X. E.; Tang, B.; Taylor, R.; Williams, A.; XiangKuan, J.; Xu, P.; Yan, Z.; Zarov, I.; Zhang, Y.; Fan, A.; Kambadur, M.; Narang, S.; Rodriguez, A.; Stojnic, R.; Edunov, S.; and Scialom, T. 2023 · 2023
Later among the works it cites.
She Elicits Requirements and He Tests: Software Engineering Gender Bias in Large Language Models
Treude, C.; and Hata, H. 2023 · 2023
Later among the works it cites.
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Wang, B.; Chen, W.; Pei, H.; Xie, C.; Kang, M.; Zhang, C.; Xu, C.; Xiong, Z.; Dutta, R.; Schaeffer, R.; Truong, S. T.; Arora, S.; Mazeika, M.; Hendrycks, D.; Lin, Z.; Cheng, Y.; Koyejo, S.; Song, D.; and Li, B. 2023 · 2023
Later among the works it cites.
A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection
Yang, S.; Sun, R.; and Wan, X. 2023 · 2023
Later among the works it cites.
Human-in-the-loop Machine Translation with Large Language Model
Yang, X.; Zhan, R.; Wong, D. F.; Wu, J.; and Chao, L. S. 2023 · 2023
Later among the works it cites.
Llm lies: Hallucinations are not bugs, but features as adversarial examples
Yao, J.-Y.; Ning, K.-P.; Liu, Z.-H.; Ning, M.-N.; and Yuan, L. 2023 · 2023
Later among the works it cites.
Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility
Ye, W.; Ou, M.; Li, T.; Ma, X.; Yanggong, Y.; Wu, S.; Fu, J.; Chen, G.; and Zhao, J. 2023 · 2023
Later among the works it cites.
Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration
Yu, P.; Xu, H.; Hu, X.; and Deng, C. 2023 · 2023
Later among the works it cites.
Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts
Zamfirescu-Pereira, J.; Wong, R. Y.; Hartmann, B.; and Yang, Q. 2023 · 2023
Later among the works it cites.
Human-in-the-loop Schema Induction
Zhang, T.; Tham, I.; Hou, Z.; Ren, J.; Zhou, L.; Xu, H.; Zhang, L.; Martin, L. J.; Dror, R.; Li, S.; Ji, H.; Palmer, M.; Brown, S. W.; Suchocki, R.; and Callison-Burch, C. 2023 · 2023
Later among the works it cites.
A study on robustness and reliability of large language model code generation
Zhong, L.; and Wang, Z. 2023 · 2023
Later among the works it cites.
From Prompt Engineering to Prompt Science With Human in the Loop
Shah, C. 2024 · 2024
Closest in time.