Fetching the paper…

Reading the bibliography…

2023

MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation

Ulrich, Constantin, Isensee, Fabian, Wald, Tassilo et al.

Original
PDF
Post on X

Understand

The medical imaging community generates a wealth of datasets, many of which are openly accessible and annotated for specific diseases and tasks such as multi-organ or lesion segmentation.

Current practices continue to limit model training and supervised pre-training to one or a few similar datasets, neglecting the synergistic potential of other available annotated data.
We propose MultiTalent, a method that leverages multiple CT datasets with diverse and conflicting class definitions to train a single model for a comprehensive structure segmentation.
Our results demonstrate improved segmentation performance compared to previous related approaches, systematically, also compared to single dataset training using state-of-the-art methods, especially for lesion segmentation and other challenging structures.

MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation · Around

Built on

Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., Moore, S., Phillips, S., Maffitt, D., Pringle, M., Tarbox, L., Prior, F.: The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. Journal of Digital Imaging pp. 1045–1057 (2013)
2013
Earlier work this paper cites.
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science, Springer International Publishing (2015)
2015
Earlier work this paper cites.
Roth, H.R., Lu, L., Farag, A., Shin, H.C., Liu, J., Turkbey, E., Summers, R.M.: Deeporgan: Multi-level deep convolutional networks for automated pancreas segmentation. arXiv:1506.06448 (2015)
Original
2015
Earlier work this paper cites.
Roth, H.R., Lu, L., Farag, A., Shin, H.C., Liu, J., Turkbey, E.B., Summers, R.M.: Deeporgan: Multi-level deep convolutional networks for automated pancreas segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. pp. 556–564. Springer International Publishing (2015)
2015
Earlier work this paper cites.
Gibson, E., Giganti, F., Hu, Y., Bonmati, E., Bandula, S., Gurusamy, K., Davidson, B., Pereira, S.P., Clarkson, M.J., Barratt, D.C.: Automatic multi-organ segmentation on abdominal ct with dense v-networks. IEEE Transactions on Medical Imaging 37
2018
Earlier work this paper cites.
Chen, S., Ma, K., Zheng, Y.: Med3d: Transfer learning for 3d medical image analysis. arXiv:1904.00625 (2019)
Original
2019
Earlier work this paper cites.
Dmitriev, K., Kaufman, A.E.: Learning multi-class segmentations from single-class datasets. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
2019
Earlier work this paper cites.
Isensee, F., Maier-Hein, K.H.: An attempt at beating the 3d u-net. arXiv1908.02182 (2019)
2019
Earlier work this paper cites.
Lambert, Z., Petitjean, C., Dubray, B., Ruan, S.: Segthor: Segmentation of thoracic organs at risk in ct images. arXiv:1912.05950 (2019)
Original
2019
Earlier work this paper cites.
Roulet, N., Slezak, D.F., Ferrante, E.: Joint learning of brain lesion and anatomy segmentation from heterogeneous datasets. In: Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning (2019)
2019
Earlier work this paper cites.
Zhou, Y., Li, Z., Bai, S., Chen, X., Han, M., Wang, C., Fishman, E., Yuille, A.: Prior-aware neural network for partially-supervised multi-organ segmentation. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019)
2019
Earlier work this paper cites.

Similar

Fang, X., Yan, P.: Multi-organ segmentation over partially labeled datasets with multi-scale feature abstraction. IEEE Transactions on Medical Imaging (2020)
2020
Cited alongside, same era.
Heller, N., Sathianathen, N., Kalapara, A., Walczak, E., Moore, K., Kaluzniak, H., Rosenberg, J., Blake, P., Rengel, Z., Oestreich, M., Dean, J., Tradewell, M., Shah, A., Tejpaul, R., Edgerton, Z., Peterson, M., Raza, S., Regmi, S., Papanikolopoulos, N., Weight, C.: The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes. arXiv:1904.00445 (2020)
Original
2020
Cited alongside, same era.
Huang, R., Zheng, Y., Hu, Z., Zhang, S., Li, H.: Multi-organ segmentation via co-training weight-averaged models from few-organ datasets. arXiv:2008.07149 (2020)
Original
2020
Cited alongside, same era.
Feng, S., Zhou, Y., Zhang, X., Zhang, Y., Wang, Y.: Ms-kd: Multi-organ segmentation with multiple binary-labeled datasets. arXiv:2108.02559 (2021)
Original
2021
Cited alongside, same era.

Then

Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H.R., Xu, D.: Unetr: Transformers for 3d medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 574–584 (January 2022)
2022
Later among the works it cites.
Isensee, F., Ulrich, C., Wald, T., Maier-Hein, K.H.: Extending nnu-net is all you need. arXiv preprint arXiv:2208.10791 (2022)
Original
2022
Later among the works it cites.
Ji, Y., Bai, H., Yang, J., Ge, C., Zhu, Y., Zhang, R., Li, Z., Zhang, L., Ma, W., Wan, X., et al.: Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. arXiv:2206.08023 (2022)
Original
2022
Later among the works it cites.
Kumar, A., Raghunathan, A., Jones, R., Ma, T., Liang, P.: Fine-tuning can distort pretrained features and underperform out-of-distribution. arxiv.2202.10054 (2022)
2022
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Fidon, L., Aertsen, M., Emam, D., Mufti, N., Guffens, F., Deprest, T., Demaerel, P., David, A.L., Melbourne, A., et al.: Label-set loss functions for partial supervision: Application to fetal brain 3d MRI parcellation. In: Medical Image Computing and Computer Assisted Intervention (2021)

2021

Cited alongside, same era.

Filbrandt, G., Kamnitsas, K., Bernstein, D., Taylor, A., Glocker, B.: Learning from partially overlapping labels: Image segmentation under annotation shift. In: Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health. Springer International Publishing (2021)

2021

Cited alongside, same era.

Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2)

2021

Cited alongside, same era.

Shi, G., Xiao, L., Chen, Y., Zhou, S.K.: Marginal loss and exclusion loss for partially supervised multi-organ segmentation. Medical Image Analysis 70

2021

Cited alongside, same era.

Zhang, J., Xie, Y., Xia, Y., Shen, C.: Dodnet: Learning to segment multi-organ and tumors from multiple partially labeled datasets. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2021)

2021

Cited alongside, same era.

Zhou, Z., Sodha, V., Pang, J., Gotway, M.B., Liang, J.: Models genesis. Medical Image Analysis (2021)

2021

Cited alongside, same era.

Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., Ronneberger, O., Summers, R.M., et al: The medical segmentation decathlon. Nature Communications (2022)

2022

Cited alongside, same era.

Landman, B., Xu, Z., Igelsias, J.E., Styner, M., Langerak, T., Klein, A.: Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge (2015), https://www.synapse.org/#!Synapse:syn3193805/wiki/217760 (25/02/2022)

2022

Later among the works it cites.

Li, H., Zhou, J., Deng, J., Chen, M.: Automatic structure segmentation for radiotherapy planning challenge (2019), https://structseg2019.grand-challenge.org/ 25/02/2022)

2022

Later among the works it cites.

Liu, P., Zheng, G.: Context-aware voxel-wise contrastive learning for label efficient multi-organ segmentation. In: Medical Image Computing and Computer Assisted Intervention (2022)

2022

Later among the works it cites.

Tang, Y., Yang, D., Li, W., Roth, H.R., Landman, B., Xu, D., Nath, V., Hatamizadeh, A.: Self-supervised pre-training of swin transformers for 3d medical image analysis. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

2022

Later among the works it cites.

Wasserthal, J., Meyer, M., Breit, H.C., Cyriac, J., Yang, S., Segeroth, M.: Totalsegmentator: robust segmentation of 104 anatomical structures in ct images. arXiv:2208.05868 (2022)

2022

Later among the works it cites.

Li, S., Wang, H., Meng, Y., Zhang, C., Song, Z.: Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation (2023)

2023

Closest in time.

Liu, J., Zhang, Y., Chen, J.N., Xiao, J., Lu, Y., Landman, B.A., Yuan, Y., Yuille, A., Tang, Y., Zhou, Z.: Clip-driven universal model for organ segmentation and tumor detection. arXiv:2301.00785 (2023)

2023

Closest in time.