Fetching the paper…
Reading the bibliography…
Recently, numerous new benchmarks have been established to evaluate the performance of large language models (LLMs) via either computing a holistic score or employing another LLM as a judge.
The Proof and Measurement of Association between Two Things
Spearman, C. 1904 · 1904
Earlier work this paper cites.
A NEW MEASURE OF RANK CORRELATION
KENDALL, M. G. 1938 · 1938
Earlier work this paper cites.
Measuring Massive Multitask Language Understanding
Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2021 · 2009
Earlier work this paper cites.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Bai, Y.; Jones, A.; Ndousse, K.; Askell, A.; Chen, A.; DasSarma, N.; Drain, D.; Fort, S.; Ganguli, D.; Henighan, T.; Joseph, N.; Kadavath, S.; Kernion, J.; Conerly, T.; El-Showk, S.; Elhage, N.; Hatfield-Dodds, Z.; Hernandez, D.; Hume, T.; Johnston, S.; Kravec, S.; Lovitt, L.; Nanda, N.; Olsson, C.; Amodei, D.; Brown, T.; Clark, J.; McCandlish, S.; Olah, C.; Mann, B.; and Kaplan, J. 2022 · 2022
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C. L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P.; Leike, J.; and Lowe, R. 2022 · 2022
Earlier work this paper cites.
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Suzgun, M.; Scales, N.; Schärli, N.; Gehrmann, S.; Tay, Y.; Chung, H. W.; Chowdhery, A.; Le, Q. V.; Chi, E. H.; Zhou, D.; and Wei, J. 2022 · 2022
Earlier work this paper cites.
Glm-130b: An open bilingual pre-trained model
Zeng, A.; Liu, X.; Du, Z.; Wang, Z.; Lai, H.; Ding, M.; Yang, Z.; Xu, Y.; Zheng, W.; Xia, X.; et al. 2022 · 2022
Earlier work this paper cites.
Benchmarking Foundation Models with Language-Model-as-an-Examiner
Bai, Y.; Ying, J.; Cao, Y.; Lv, X.; He, Y.; Wang, X.; Yu, J.; Zeng, K.; Xiao, Y.; Lyu, H.; Zhang, J.; Li, J.; and Hou, L. 2023 · 2023
Earlier work this paper cites.
A Survey on Evaluation of Large Language Models
Chang, Y.; Wang, X.; Wang, J.; Wu, Y.; Yang, L.; Zhu, K.; Chen, H.; Yi, X.; Wang, C.; Wang, Y.; Ye, W.; Zhang, Y.; Chang, Y.; Yu, P. S.; Yang, Q.; and Xie, X. 2023 · 2023
Earlier work this paper cites.
Claude2-Alpaca: Instruction tuning datasets distilled from claude
Chen, L.; Saifullah, K.; Li, M.; Zhou, T.; and Huang, H. 2023 · 2023
Earlier work this paper cites.
INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models
Chia, Y. K.; Hong, P.; Bing, L.; and Poria, S. 2023 · 2023
Earlier work this paper cites.
Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality
Chiang, W.-L.; Li, Z.; Lin, Z.; Sheng, Y.; Wu, Z.; Zhang, H.; Zheng, L.; Zhuang, S.; Zhuang, Y.; Gonzalez, J. E.; Stoica, I.; and Xing, E. P. 2023 · 2023
Cited alongside, same era.
GPTScore: Evaluate as You Desire
Fu, J.; Ng, S.-K.; Jiang, Z.; and Liu, P. 2023 · 2023
Cited alongside, same era.
Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; de las Casas, D.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; Lavaud, L. R.; Lachaux, M.-A.; Stock, P.; Scao, T. L.; Lavril, T.; Wang, T.; Lacroix, T.; and Sayed, W. E. 2023 · 2023
Cited alongside, same era.
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Liu, Y.; Iter, D.; Xu, Y.; Wang, S.; Xu, R.; and Zhu, C. 2023 · 2023
Cited alongside, same era.
LLaMA: Open and Efficient Foundation Language Models
Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; Rodriguez, A.; Joulin, A.; Grave, E.; and Lample, G. 2023 · 2023
Later among the works it cites.
Style Over Substance: Evaluation Biases for Large Language Models
Wu, M.; and Aji, A. F. 2023 · 2023
Later among the works it cites.
WizardLM: Empowering Large Language Models to Follow Complex Instructions
Xu, C.; Sun, Q.; Zheng, K.; Geng, X.; Zhao, P.; Feng, J.; Tao, C.; and Jiang, D. 2023 · 2023
Later among the works it cites.
Rethinking Benchmark and Contamination for Language Models with Rephrased Samples
Yang, S.; Chiang, W.-L.; Zheng, L.; Gonzalez, J. E.; and Stoica, I. 2023 · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Long, J. 2023 · 2023
Cited alongside, same era.
OpenAI. 2023 · 2023
Cited alongside, same era.
Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
Saha, S.; Levy, O.; Celikyilmaz, A.; Bansal, M.; Weston, J.; and Li, X. 2023 · 2023
Cited alongside, same era.
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
Sainz, O.; Campos, J.; García-Ferrero, I.; Etxaniz, J.; de Lacalle, O. L.; and Agirre, E. 2023 · 2023
Cited alongside, same era.
Verbosity Bias in Preference Labeling by Large Language Models
Saito, K.; Wachi, A.; Wataoka, K.; and Akimoto, Y. 2023 · 2023
Cited alongside, same era.
Stanford Alpaca: An Instruction-following LLaMA model
Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023 · 2023
Cited alongside, same era.
Llama 2: Open Foundation and Fine-Tuned Chat Models
et al., H. T. 2023a
Cited in the paper.
et al., R. A. 2023b
Cited in the paper.
Zhong, W.; Cui, R.; Guo, Y.; Liang, Y.; Lu, S.; Wang, Y.; Saied, A.; Chen, W.; and Duan, N. 2023 · 2023
Later among the works it cites.
JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Zhu, L.; Wang, X.; and Wang, X. 2023 · 2023
Later among the works it cites.
ChatGPT’s One-year Anniversary: Are Open-Source Large Language Models Catching up?
Chen, H.; Jiao, F.; Li, X.; Qin, C.; Ravaut, M.; Zhao, R.; Xiong, C.; and Joty, S. 2024 · 2024
Closest in time.
TreeEval: Benchmark-Free Evaluation of Large Language Models through Tree Planning
Li, X.; Lan, Y.; and Yang, C. 2024 · 2024
Closest in time.
Secrets of RLHF in Large Language Models Part II: Reward Modeling
Wang, B.; Zheng, R.; Chen, L.; Liu, Y.; Dou, S.; Huang, C.; Shen, W.; Jin, S.; Zhou, E.; Shi, C.; Gao, S.; Xu, N.; Zhou, Y.; Fan, X.; Xi, Z.; Zhao, J.; Wang, X.; Ji, T.; Yan, H.; Shen, L.; Chen, Z.; Gui, T.; Zhang, Q.; Qiu, X.; Huang, X.; Wu, Z.; and Jiang, Y.-G. 2024 · 2024
Closest in time.
DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
Zhu, K.; Chen, J.; Wang, J.; Gong, N. Z.; Yang, D.; and Xie, X. 2024 · 2024
Closest in time.