Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement

Understand

Radiology report generation (RRG) has attracted significant attention due to its potential to reduce the workload of radiologists.

• The performance of current RRG approaches remains unsatisfactory against clinical standards.
• This paper introduces a novel RRG method, MLLM-RRG, that integrates multimodal large language models (MLLMs) with various types of clinical knowledge to generate accurate and comprehensive chest X-ray reports.
• Our method first designs a referring anatomical feature extractor that leverages anatomical knowledge to analyze different regions of the chest X-ray image and extract visual features without explicitly detecting regions.