MedICaT: A Dataset of Medical Images, Captions, and Textual References

Understand

Understanding the relationship between figures and text is key to scientific document understanding.

• Medical figures in particular are quite complex, often consisting of several subfigures (75% of figures in our dataset), with detailed text describing their content.
• Previous work studying figures in scientific papers focused on classifying figure content rather than understanding how images relate to the text.
• To address challenges in figure retrieval and figure-to-text alignment, we introduce MedICaT, a dataset of medical images in context.