Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models

Understand

Background: Despite the current ubiquity of Large Language Models (LLMs) across the medical domain, there is a surprising lack of studies which address their reasoning behaviour.

• We emphasise the importance of understanding reasoning behaviour as opposed to high-level prediction accuracies, since it is equivalent to explainable AI (XAI) in this context.
• In particular, achieving XAI in medical LLMs used in the clinical domain will have a significant impact across the healthcare sector.
• Results: Therefore, in this work, we adapt the existing concept of reasoning behaviour and articulate its interpretation within the specific context of medical LLMs.