Pan-Cancer Integrative Histology-Genomic Analysis via Interpretable Multimodal Deep Learning

Understand

The rapidly emerging field of deep learning-based computational pathology has demonstrated promise in developing objective prognostic models from histology whole slide images.

• However, most prognostic models are either based on histology or genomics alone and do not address how histology and genomics can be integrated to develop joint image-omic prognostic models.
• Additionally identifying explainable morphological and molecular descriptors from these models that govern such prognosis is of interest.
• We used multimodal deep learning to integrate gigapixel whole slide pathology images, RNA-seq abundance, copy number variation, and mutation data from 5,720 patients across 14 major cancer types.