Fetching the paper…
Reading the bibliography…
The emergence of various medical large language models (LLMs) in the medical domain has highlighted the need for unified evaluation standards, as manual evaluation of LLMs proves to be time-consuming and labor-intensive.
Statistical theory for logistic mental test models with a prior distribution of ability
Birnbaum, A. 1969 · 1969
Earlier work this paper cites.
Bleu: a method for automatic evaluation of machine translation
Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002 · 2002
Earlier work this paper cites.
Rouge: A package for automatic evaluation of summaries
Lin, C.-Y. 2004 · 2004
Earlier work this paper cites.
Measuring massive multitask language understanding
Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2020 · 2009
Earlier work this paper cites.
PubMedQA: A Dataset for Biomedical Research Question Answering
Jin, Q.; Dhingra, B.; Liu, Z.; Cohen, W.; and Lu, X. 2019 · 2019
Earlier work this paper cites.
What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams
Jin, D.; Pan, E.; Oufattole, N.; Weng, W.-H.; Fang, H.; and Szolovits, P. 2021 · 2021
Earlier work this paper cites.
MLEC-QA: A Chinese multi-choice biomedical question answering dataset
Li, J.; Zhong, S.; and Chen, K. 2021 · 2021
Earlier work this paper cites.
GLM: General Language Model Pretraining with Autoregressive Blank Infilling
Du, Z.; Qian, Y.; Liu, X.; Ding, M.; Qiu, J.; Yang, Z.; and Tang, J. 2022 · 2022
Earlier work this paper cites.
Can large language models reason about medical questions?
Liévin, V.; Hother, C. E.; and Winther, O. 2022 · 2022
Earlier work this paper cites.
Crosslingual Generalization through Multitask Finetuning
Muennighoff, N.; Wang, T.; Sutawika, L.; Roberts, A.; Biderman, S.; Scao, T. L.; Bari, M. S.; Shen, S.; Yong, Z.-X.; Schoelkopf, H.; Tang, X.; Radev, D.; Aji, A. F.; Almubarak, K.; Albanie, S.; Alyafeai, Z.; Webson, A.; Raff, E.; and Raffel, C. 2022 · 2022
Cited alongside, same era.
Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering
Pal, A.; Umapathi, L. K.; and Sankarasubbu, M. 2022 · 2022
Cited alongside, same era.
Bai, J.; Bai, S.; Chu, Y.; Cui, Z.; Dang, K.; Deng, X.; Fan, Y.; Ge, W.; Han, Y.; Huang, F.; Hui, B.; Ji, L.; Li, M.; Lin, J.; Lin, R.; Liu, D.; Liu, G.; Lu, C.; Lu, K.; Ma, J.; Men, R.; Ren, X.; Ren, X.; Tan, C.; Tan, S.; Tu, J.; Wang, P.; Wang, S.; Wang, W.; Wu, S.; Xu, B.; Xu, J.; Yang, A.; Yang, H.; Yang, J.; Yang, S.; Yao, Y.; Yu, B.; Yuan, H.; Yuan, Z.; Zhang, J.; Zhang, X.; Zhang, Y.; Zhang, Z.; Zhou, C.; Zhou, J.; Zhou, X.; and Zhu, T. 2023 · 2023
Cited alongside, same era.
A Survey on Evaluation of Large Language Models
Chang, Y.; Wang, X.; Wang, J.; Wu, Y.; Yang, L.; Zhu, K.; Chen, H.; Yi, X.; Wang, C.; Wang, Y.; Ye, W.; Zhang, Y.; Chang, Y.; Yu, P. S.; Yang, Q.; and Xie, X. 2023 · 2023
Capabilities of GPT-4 on Medical Challenge Problems
Nori, H.; King, N.; McKinney, S. M.; Carignan, D.; and Horvitz, E. 2023 · 2023
Closest in time.
OpenAI. 2023 · 2023
Closest in time.
HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge
Wang, H.; Liu, C.; Xi, N.; Qiang, Z.; Zhao, S.; Qin, B.; and Liu, T. 2023 · 2023
Closest in time.
PMC-LLaMA: Towards Building Open-source Language Models for Medicine
Wu, C.; Lin, W.; Zhang, X.; Zhang, Y.; Wang, Y.; and Xie, W. 2023 · 2023
Closest in time.
Sunsimiao: Chinese Medicine LLM
Xin Yan, D. X. 2023 · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Chen, Y.; Wang, Z.; Xing, X.; huimin zheng; Xu, Z.; Fang, K.; Wang, J.; Li, S.; Wu, J.; Liu, Q.; and Xu, X. 2023 · 2023
Cited alongside, same era.
MedAlpaca – An Open-Source Collection of Medical Conversational AI Models and Training Data
Han, T.; Adams, L. C.; Papaioannou, J.-M.; Grundmann, P.; Oberhauser, T.; Löser, A.; Truhn, D.; and Bressem, K. K. 2023 · 2023
Cited alongside, same era.
Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models
Kung, T. H.; Cheatham, M.; Medenilla, A.; Sillos, C.; De Leon, L.; Elepaño, C.; Madriaga, M.; Aggabao, R.; Diaz-Candido, G.; Maningo, J.; and Tseng, V. 2023 · 2023
Cited alongside, same era.
ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge
Li, Y.; Li, Z.; Zhang, K.; Dan, R.; Jiang, S.; and Zhang, Y. 2023 · 2023
Cited alongside, same era.
Benchmarking Large Language Models on CMExam–A Comprehensive Chinese Medical Exam Dataset
Liu, J.; Zhou, P.; Hua, Y.; Chong, D.; Tian, Z.; Liu, A.; Wang, H.; You, C.; Guo, Z.; Zhu, L.; et al. 2023 · 2023
Cited alongside, same era.
Large language models encode clinical knowledge
Singhal, K.; Azizi, S.; Tu, T.; Mahdavi, S. S.; Wei, J.; Chung, H. W.; Scales, N.; Tanwani, A.; Cole-Lewis, H.; Pfohl, S.; Payne, P.; Seneviratne, M.; Gamble, P.; Kelly, C.; Babiker, A.; Schärli, N.; Chowdhery, A.; Mansfield, P.; Demner-Fushman, D.; Agüera y Arcas, B.; Webster, D.; Corrado, G. S.; Matias, Y.; Chou, K.; Gottweis, J.; Tomasev, N.; Liu, Y.; Rajkomar, A.; Barral, J.; Semturs, C.; Karthikesalingam, A.; and Natarajan, V. 2023a
Cited in the paper.
Large language models encode clinical knowledge
Singhal, K.; Azizi, S.; Tu, T.; Mahdavi, S. S.; Wei, J.; Chung, H. W.; Scales, N.; Tanwani, A.; Cole-Lewis, H.; Pfohl, S.; et al. 2023b
Cited in the paper.
MedGPTEval: A Dataset and Benchmark to Evaluate Responses of Large Language Models in Medicine
Xu, J.; Lu, L.; Yang, S.; Liang, B.; Peng, X.; Pang, J.; Ding, J.; Shi, X.; Yang, L.; Song, H.; Li, K.; Sun, X.; and Zhang, S. 2023b
Cited in the paper.
DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task
Xiong, H.; Wang, S.; Zhu, Y.; Zhao, Z.; Liu, Y.; Huang, L.; Wang, Q.; and Shen, D. 2023 · 2023
Closest in time.
Baize: An open-source chat model with parameter-efficient tuning on self-chat data
Xu, C.; Guo, D.; Duan, N.; and McAuley, J. 2023a · 2023
Closest in time.
Glm-130b: An open bilingual pre-trained model
Zeng, A.; Liu, X.; Du, Z.; Wang, Z.; Lai, H.; Ding, M.; Yang, Z.; Xu, Y.; Zheng, W.; Xia, X.; et al. 2023 · 2023
Closest in time.
ChatMed: A Chinese Medical Large Language Model
Zhu, W.; and Wang, X. 2023 · 2023
Closest in time.