Benchmarking large language models for biomedical natural language processing applications and recommendations

Understand

The rapid growth of biomedical literature poses challenges for manual knowledge curation and synthesis.

• Biomedical Natural Language Processing (BioNLP) automates the process.
• While Large Language Models (LLMs) have shown promise in general domains, their effectiveness in BioNLP tasks remains unclear due to limited benchmarks and practical guidelines.
• We perform a systematic evaluation of four LLMs, GPT and LLaMA representatives on 12 BioNLP benchmarks across six applications.