Can GPT Redefine Medical Understanding? Evaluating GPT on Biomedical Machine Reading Comprehension

Understand

Large language models (LLMs) have shown remarkable performance on many tasks in different domains.

• However, their performance in closed-book biomedical machine reading comprehension (MRC) has not been evaluated in depth.
• In this work, we evaluate GPT on four closed-book biomedical MRC benchmarks.
• We experiment with different conventional prompting techniques as well as introduce our own novel prompting method.