Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition

BERTScore: Evaluating text generation with BERT

Original

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019 · 1904

Earlier work this paper cites.

Codesearchnet challenge: Evaluating the state of semantic code search

Original

Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019 · 1909

Earlier work this paper cites.

The Rating of Chessplayers: Past and Present

Arpad E Elo and Sam Sloan. 1978 · 1978

Earlier work this paper cites.

JudgeLM: Fine-tuned large language models are scalable judges

Original

Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023 · 1978

Earlier work this paper cites.

BLEU: A method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

Goodhart’s law and performance indicators in higher education

Lewis Elton. 2004 · 2004

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Maximum differentiation (MAD) competition: A methodology for comparing computational models of perceptual quantities

Zhou Wang and Eero P Simoncelli. 2008 · 2008

Earlier work this paper cites.

Measuring massive multitask language understanding

Original

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020 · 2009

Earlier work this paper cites.

A survey on contrastive self-supervised learning

Original

Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. 2020 · 2011

Earlier work this paper cites.

Why we need new evaluation metrics for NLG

Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, and Verena Rieser. 2017 · 2017

Earlier work this paper cites.

Variational adversarial active learning

Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. 2019 · 2019

Earlier work this paper cites.

A general language assistant as a laboratory for alignment

Original

Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. 2021 · 2021

Earlier work this paper cites.

Program synthesis with large language models

Original

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021 · 2021

Earlier work this paper cites.

Training verifiers to solve math word problems

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021 · 2021

Earlier work this paper cites.

GLM: General language model pretraining with autoregressive blank infilling

Original

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2021 · 2021

Earlier work this paper cites.

Dynabench: Rethinking benchmarking in NLP

Original

Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, et al. 2021 · 2021

Earlier work this paper cites.

Cross-task generalization via natural language crowdsourcing instructions

Original

Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021 · 2021

Earlier work this paper cites.

Adversarial GLUE: A multi-task benchmark for robustness evaluation of language models

Original

Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. 2021 · 2021

Earlier work this paper cites.

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022 · 2022

Earlier work this paper cites.

Beyond the imitation game: Quantifying and extrapolating the capabilities of language models

Original

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022 · 2022

Earlier work this paper cites.

Self-Instruct: Aligning language models with self-generated instructions

Original

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022 · 2022

Cited alongside, same era.

An in-depth look at Gemini’s language abilities

Original

Syeda Nahida Akter, Zichun Yu, Aashiq Muhamed, Tianyue Ou, Alex Bäuerle, Ángel Alexander Cabrera, Krish Dholakia, Chenyan Xiong, and Graham Neubig. 2023 · 2023

Cited alongside, same era.

Qwen technical report

Original

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023 · 2023

Cited alongside, same era.

Which prompts make the difference? Data prioritization for efficient human LLM evaluation

Original

Meriem Boubdir, Edward Kim, Beyza Ermis, Marzieh Fadaee, and Sara Hooker. 2023 · 2023

Cited alongside, same era.

LLaMA: Open and efficient foundation language models

Original

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 · 2023

Later among the works it cites.

Anchor points: Benchmarking models with much fewer examples

Original

Rajan Vivek, Kawin Ethayarajh, Diyi Yang, and Douwe Kiela. 2023 · 2023

Later among the works it cites.

WizardLM: Empowering large language models to follow complex instructions

Original

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023 · 2023

Later among the works it cites.

Evaluating large language models at evaluating instruction following

Original

Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2023 · 2023

Later among the works it cites.