Textual Inversion and Self-supervised Refinement for Radiology Report Generation

Understand

Existing mainstream approaches follow the encoder-decoder paradigm for generating radiology reports.

• They focus on improving the network structure of encoders and decoders, which leads to two shortcomings: overlooking the modality gap and ignoring report content constraints.
• In this paper, we proposed Textual Inversion and Self-supervised Refinement (TISR) to address the above two issues.
• Specifically, textual inversion can project text and image into the same space by representing images as pseudo words to eliminate the cross-modeling gap.