DrBenchmark: A Large Language Understanding Evaluation Benchmark for French Biomedical Domain

Understand

The biomedical domain has sparked a significant interest in the field of Natural Language Processing (NLP), which has seen substantial advancements with pre-trained language models (PLMs).

• However, comparing these models has proven challenging due to variations in evaluation protocols across different models.
• A fair solution is to aggregate diverse downstream tasks into a benchmark, allowing for the assessment of intrinsic PLMs qualities from various perspectives.
• Although still limited to few languages, this initiative has been undertaken in the biomedical field, notably English and Chinese.