MedBench: A Comprehensive, Standardized, and Reliable Benchmarking System for Evaluating Chinese Medical Large Language Models

Understand

Ensuring the general efficacy and goodness for human beings from medical large language models (LLM) before real-world deployment is crucial.

• However, a widely accepted and accessible evaluation process for medical LLM, especially in the Chinese context, remains to be established.
• In this work, we introduce "MedBench", a comprehensive, standardized, and reliable benchmarking system for Chinese medical LLM.
• First, MedBench assembles the currently largest evaluation dataset (300,901 questions) to cover 43 clinical specialties and performs multi-facet evaluation on medical LLM.