Measuring Massive Multitask Chinese Understanding

Understand

The development of large-scale Chinese language models is flourishing, yet there is a lack of corresponding capability assessments.

• Therefore, we propose a test to measure the multitask accuracy of large Chinese language models.
• This test encompasses four major domains, including medicine, law, psychology, and education, with 15 subtasks in medicine and 8 subtasks in education.
• We found that the best-performing models in the zero-shot setting outperformed the worst-performing models by nearly 18.6 percentage points on average.