MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering

Understand

Large Language Models (LLMs) have the potential of facilitating the development of Artificial Intelligence technology to assist medical experts for interactive decision support, which has been demonstrated by their competitive performances in Medical QA.

• However, while impressive, the required quality bar for medical applications remains far from being achieved.
• Currently, LLMs remain challenged by outdated knowledge and by their tendency to generate hallucinated content.
• Furthermore, most benchmarks to assess medical knowledge lack reference gold explanations which means that it is not possible to evaluate the reasoning of LLMs predictions.