2023

Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets

He, Sheng, Bao, Rina, Li, Jingpeng et al.

Understand

Background: The segment-anything model (SAM), introduced in April 2023, shows promise as a benchmark model and a universal solution to segment various natural images.

It comes without previously-required re-training or fine-tuning specific to each new dataset.
Purpose: To test SAM's accuracy in various medical image segmentation tasks and investigate potential factors that may affect its accuracy in medical images.
Methods: SAM was tested on 12 public medical image segmentation datasets involving 7,451 subjects.

Built on

E. Peli, “Contrast in complex images,” JOSA A , vol. 7, no. 10, pp. 2032–2040, 1990
1990
Earlier work this paper cites.
B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, J. Slotboom, R. Wiest et al. , “The multimodal brain tumor image segmentation benchmark (BRATS),” IEEE transactions on medical imaging , vol. 34, no. 10, pp. 1993–2024, 2014
2014
Earlier work this paper cites.
S. Jaeger, S. Candemir, S. Antani, Y.-X. J. Wáng, P.-X. Lu, and G. Thoma, “Two public chest x-ray datasets for computer-aided screening of pulmonary diseases,” Quantitative imaging in medicine and surgery , vol. 4, no. 6, p. 475, 2014
2014
Earlier work this paper cites.
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 . Springer, 2015, pp. 234–241
2015
Earlier work this paper cites.
S. Bakas, H. Akbari, A. Sotiras, M. Bilello, M. Rozycki, J. S. Kirby, J. B. Freymann, K. Farahani, and C. Davatzikos, “Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features,” Scientific data , vol. 4, no. 1, pp. 1–13, 2017
2017
Earlier work this paper cites.
O. Bernard, A. Lalande, C. Zotti, F. Cervenansky, X. Yang, P.-A. Heng, I. Cetin, K. Lekadir, O. Camara, M. A. G. Ballester et al. , “Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved?” IEEE transactions on medical imaging , vol. 37, no. 11, pp. 2514–2525, 2018
2018
Earlier work this paper cites.
S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, R. T. Shinohara, C. Berger, S. M. Ha, M. Rozycki et al. , “Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge,” arXiv preprint arXiv:1811.02629 , 2018
Original
2018
Earlier work this paper cites.
M. A. Attiyeh, J. Chakraborty, A. Doussot, L. Langdon-Embry, S. Mainarich, M. Gönen, V. P. Balachandran, M. I. D’Angelica, R. P. DeMatteo, W. R. Jarnagin et al. , “Survival prediction in pancreatic ductal adenocarcinoma by quantitative computed tomography image analysis,” Annals of surgical oncology , vol. 25, no. 4, pp. 1034–1042, 2018
2018
Earlier work this paper cites.
P. Tschandl, C. Rosendahl, and H. Kittler, “The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Scientific data , vol. 5, no. 1, pp. 1–9, 2018
2018
Earlier work this paper cites.
V. Remeš and M. Haindl, “Region of interest contrast measures,” Kybernetika , vol. 54, no. 5, pp. 978–990, 2018
2018
Earlier work this paper cites.
Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation,” IEEE transactions on medical imaging , vol. 39, no. 6, pp. 1856–1867, 2019
2019
Earlier work this paper cites.
M. H. Hesamian, W. Jia, X. He, and P. Kennedy, “Deep learning techniques for medical image segmentation: achievements and challenges,” Journal of digital imaging , vol. 32, pp. 582–596, 2019
2019
Earlier work this paper cites.

Similar

N. Codella, V. Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, B. Helba, A. Kalloo, K. Liopyris, M. Marchetti et al. , “Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic),” arXiv preprint arXiv:1902.03368 , 2019
Original
2019
Cited alongside, same era.
J. Schlemper, O. Oktay, M. Schaap, M. Heinrich, B. Kainz, B. Glocker, and D. Rueckert, “Attention gated networks: Learning to leverage salient regions in medical images,” Medical image analysis , vol. 53, pp. 197–207, 2019
2019
Cited alongside, same era.
W. Al-Dhabyani, M. Gomaa, H. Khaled, and A. Fahmy, “Dataset of breast ultrasound images,” Data in brief , vol. 28, p. 104863, 2020
2020
Cited alongside, same era.
D. Jha, P. H. Smedsrud, M. A. Riegler, P. Halvorsen, T. d. Lange, D. Johansen, and H. D. Johansen, “Kvasir-seg: A segmented polyp dataset,” in International Conference on Multimedia Modeling . Springer, 2020, pp. 451–462
2020
Cited alongside, same era.
Q. Liu, Q. Dou, L. Yu, and P. A. Heng, “MS-Net: multi-site network for improving prostate segmentation with heterogeneous mri data,” IEEE transactions on medical imaging , vol. 39, no. 9, pp. 2713–2724, 2020
2020
Cited alongside, same era.
X. Liu, L. Song, S. Liu, and Y. Zhang, “A review of deep-learning-based medical image segmentation methods,” Sustainability , vol. 13, no. 3, p. 1224, 2021
2021
Cited alongside, same era.
S. K. Zhou, H. Greenspan, C. Davatzikos, J. S. Duncan, B. Van Ginneken, A. Madabhushi, J. L. Prince, D. Rueckert, and R. M. Summers, “A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises,” Proceedings of the IEEE , vol. 109, no. 5, pp. 820–838, 2021
2021
Cited alongside, same era.
Z. Xiong, Q. Xia, Z. Hu, N. Huang, C. Bian, Y. Zheng, S. Vesal, N. Ravikumar, A. Maier, X. Yang et al. , “A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging,” Medical image analysis , vol. 67, p. 101832, 2021
2021
Cited alongside, same era.
J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” arXiv preprint arXiv:2102.04306 , 2021
Original
2021
Cited alongside, same era.
W. Choi, N. Dahiya, and S. Nadeem, “CIRDataset: A large-scale dataset for clinically-interpretable lung nodule radiomics and malignancy prediction,” in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2022, pp. 13–22
2022
Cited alongside, same era.
M. Antonelli, A. Reinke, S. Bakas, K. Farahani, A. Kopp-Schneider, B. A. Landman, G. Litjens, B. Menze, O. Ronneberger, R. M. Summers et al. , “The medical segmentation decathlon,” Nature communications , vol. 13, no. 1, pp. 1–13, 2022
2022
Cited alongside, same era.
H. Wang, P. Cao, J. Wang, and O. R. Zaiane, “Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer,” in Proceedings of the AAAI conference on artificial intelligence , vol. 36, no. 3, 2022, pp. 2441–2449
2022
Cited alongside, same era.

Then

S. He, R. Bao, P. E. Grant, and Y. Ou, “U-netmer: U-net meets transformer for medical image segmentation,” arXiv preprint arXiv:2304.01401 , 2023
Original
2023
Closest in time.
A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo et al. , “Segment anything,” arXiv preprint arXiv:2304.02643 , 2023
Original
2023
Closest in time.
P. Bilic, P. Christ, H. B. Li, E. Vorontsov, A. Ben-Cohen, G. Kaissis, A. Szeskin, C. Jacobs, G. E. H. Mamani, G. Chartrand et al. , “The liver tumor segmentation benchmark (LiTS),” Medical Image Analysis , vol. 84, p. 102680, 2023
2023
Closest in time.
S. He, Y. Feng, P. E. Grant, and Y. Ou, “Segmentation ability map: Interpret deep features for medical image segmentation,” Medical Image Analysis , vol. 84, p. 102726, 2023
2023
Closest in time.
C. Hu and X. Li, “When sam meets medical images: An investigation of segment anything model (sam) on multi-phase liver tumor segmentation,” arXiv preprint arXiv:2304.08506 , 2023
Original
2023
Closest in time.
T. Zhou, Y. Zhang, Y. Zhou, Y. Wu, and C. Gong, “Can sam segment polyps?” arXiv preprint arXiv:2304.07583 , 2023
Original
2023
Closest in time.
R. Deng, C. Cui, Q. Liu, T. Yao, L. W. Remedios, S. Bao, B. A. Landman, L. E. Wheless, L. A. Coburn, K. T. Wilson et al. , “Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,” arXiv preprint arXiv:2304.04155 , 2023
Original
2023
Closest in time.
S. Mohapatra, A. Gosai, and G. Schlaug, “Brain extraction comparing segment anything model (sam) and fsl brain extraction tool,” arXiv preprint arXiv:2304.04738 , 2023
Original
2023
Closest in time.
D. Cheng, Z. Qin, Z. Jiang, S. Zhang, Q. Lao, and K. Li, “Sam on medical images: A comprehensive study on three prompt modes,” arXiv preprint arXiv:2305.00035 , 2023
Original
2023
Closest in time.
J. Ma and B. Wang, “Segment anything in medical images,” arXiv preprint arXiv:2304.12306 , 2023
Original
2023
Closest in time.
T. Chen, L. Zhu, C. Ding, R. Cao, S. Zhang, Y. Wang, Z. Li, L. Sun, P. Mao, and Y. Zang, “Sam fails to segment anything?–sam-adapter: Adapting sam in underperformed scenes: Camouflage, shadow, and more,” arXiv preprint arXiv:2304.09148 , 2023
Original
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…