AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator

Understand

Artificial intelligence has significantly advanced healthcare, particularly through large language models (LLMs) that excel in medical question answering benchmarks.

• However, their real-world clinical application remains limited due to the complexities of doctor-patient interactions.
• To address this, we introduce \textbf{AI Hospital}, a multi-agent framework simulating dynamic medical interactions between \emph{Doctor} as player and NPCs including \emph{Patient}, \emph{Examiner}, \emph{Chief Physician}.
• This setup allows for realistic assessments of LLMs in clinical scenarios.