2024

VM-UNet: Vision Mamba UNet for Medical Image Segmentation

Ruan, Jiacheng, Li, Jincheng, Xiang, Suncheng

Understand

In the realm of medical image segmentation, both CNN-based and Transformer-based models have been extensively explored.

However, CNNs exhibit limitations in long-range modeling capabilities, whereas Transformers are hampered by their quadratic computational complexity.
Recently, State Space Models (SSMs), exemplified by Mamba, have emerged as a promising approach.
They not only excel in modeling long-range interactions but also maintain a linear computational complexity.

Built on

R. E. Kalman, “A new approach to linear filtering and prediction problems,” 1960
1960
Earlier work this paper cites.
S. Sun, D. R. Haynor, and Y. Kim, “Semiautomatic video object segmentation using vsnakes,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 13, no. 1, pp. 75–82, 2003
2003
Earlier work this paper cites.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” The journal of machine learning research , vol. 15, no. 1, pp. 1929–1958, 2014
2014
Earlier work this paper cites.
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention . Springer, 2015, pp. 234–241
2015
Earlier work this paper cites.
B. Landman, Z. Xu, J. Igelsias, M. Styner, T. Langerak, and A. Klein, “Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge,” in Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge , vol. 5, 2015, p. 12
2015
Earlier work this paper cites.
Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3d u-net: learning dense volumetric segmentation from sparse annotation,” in International conference on medical image computing and computer-assisted intervention . Springer, 2016, pp. 424–432
2016
Earlier work this paper cites.
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450 , 2016
Original
2016
Earlier work this paper cites.
F. Milletari, N. Navab, and S.-A. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” in 2016 fourth international conference on 3D vision (3DV) . IEEE, 2016, pp. 565–571
2016
Earlier work this paper cites.
Loshchilov, Ilya and Hutter, Frank, “Sgdr: Stochastic gradient descent with warm restarts,” arXiv preprint arXiv:1608.03983 , 2016
Original
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
M. Berseth, “Isic 2017-skin lesion analysis towards melanoma detection,” arXiv preprint arXiv:1703.00523 , 2017
Original
2017
Earlier work this paper cites.
https://challenge.isic-archive.com/data/#2017
2017
Earlier work this paper cites.
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101 , 2017
Original
2017
Earlier work this paper cites.
Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in Deep learning in medical image analysis and multimodal learning for clinical decision support . Springer, 2018, pp. 3–11
2018
Earlier work this paper cites.
O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz et al. , “Attention u-net: Learning where to look for the pancreas,” arXiv preprint arXiv:1804.03999 , 2018
Original
2018
Earlier work this paper cites.
S. Elfwing, E. Uchibe, and K. Doya, “Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,” Neural networks , vol. 107, pp. 3–11, 2018
2018
Earlier work this paper cites.

Similar

https://challenge.isic-archive.com/data/#2018
2018
Cited alongside, same era.
N. Codella, V. Rotemberg, P. Tschandl, M. E. Celebi, S. Dusza, D. Gutman, B. Helba, A. Kalloo, K. Liopyris, M. Marchetti et al. , “Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic),” arXiv preprint arXiv:1902.03368 , 2019
Original
2019
Cited alongside, same era.
M. Z. Alom, C. Yakopcic, M. Hasan, T. M. Taha, and V. K. Asari, “Recurrent residual u-net for medical image segmentation,” Journal of Medical Imaging , vol. 6, no. 1, pp. 014 006–014 006, 2019
2019
Cited alongside, same era.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020
Original
2020
Cited alongside, same era.
H. Seo, M. Badiei Khuzani, V. Vasudevan, C. Huang, H. Ren, R. Xiao, X. Jia, and L. Xing, “Machine learning techniques for biomedical image segmentation: an overview of technical aspects and introduction to state-of-art applications,” Medical physics , vol. 47, no. 5, pp. e148–e167, 2020
2020
Cited alongside, same era.
A. Gu, T. Dao, S. Ermon, A. Rudra, and C. Ré, “Hippo: Recurrent memory with optimal polynomial projections,” Advances in neural information processing systems , vol. 33, pp. 1474–1487, 2020
2020
Cited alongside, same era.
J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, and Y. Zhou, “Transunet: Transformers make strong encoders for medical image segmentation,” arXiv preprint arXiv:2102.04306 , 2021
Original
2021
Cited alongside, same era.
Y. Zhang, H. Liu, and Q. Hu, “Transfuse: Fusing transformers and cnns for medical image segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2021, pp. 14–24
2021
Cited alongside, same era.
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 10 012–10 022
2021
Cited alongside, same era.
A. Gu, K. Goel, and C. Ré, “Efficiently modeling long sequences with structured state spaces,” arXiv preprint arXiv:2111.00396 , 2021
Original
2021
Cited alongside, same era.
A. Gu, I. Johnson, K. Goel, K. Saab, T. Dao, A. Rudra, and C. Ré, “Combining recurrent, convolutional, and continuous-time models with linear state space layers,” Advances in neural information processing systems , vol. 34, pp. 572–585, 2021
2021
Cited alongside, same era.
J. Wei, Y. Hu, R. Zhang, Z. Li, S. K. Zhou, and S. Cui, “Shallow attention network for polyp segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2021, pp. 699–708
2021
Cited alongside, same era.

Then

Y. Gao, M. Zhou, D. Liu, and D. Metaxas, “A multi-scale transformer for medical image segmentation: Architectures, model efficiency, and benchmarks,” arXiv preprint arXiv:2203.00131 , 2022
Original
2022
Later among the works it cites.
R. Azad, M. T. Al-Antary, M. Heidari, and D. Merhof, “Transnorm: Transformer provides a strong spatial normalization mechanism for a deep segmentation model,” IEEE Access , vol. 10, pp. 108 205–108 215, 2022
2022
Later among the works it cites.
R. Azad, M. Heidari, M. Shariatnia, E. K. Aghdam, S. Karimijafarbigloo, E. Adeli, and D. Merhof, “Transdeeplab: Convolution-free transformer-based deeplab v3+ for medical image segmentation,” in International Workshop on PRedictive Intelligence In MEdicine . Springer, 2022, pp. 91–102
2022
Later among the works it cites.
H. Wang, P. Cao, J. Wang, and O. R. Zaiane, “Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 3, 2022, pp. 2441–2449
2022
Later among the works it cites.
H. Wang, S. Xie, L. Lin, Y. Iwamoto, X.-H. Han, Y.-W. Chen, and R. Tong, “Mixed transformer u-net for medical image segmentation,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 2390–2394
2022
Later among the works it cites.
J. Ruan, M. Xie, J. Gao, T. Liu, and Y. Fu, “Ege-unet: an efficient group enhanced unet for skin lesion segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2023, pp. 481–490
2023
Later among the works it cites.
H. Li, D.-H. Zhai, and Y. Xia, “Erdunet: An efficient residual double-coding unet for medical image segmentation,” IEEE Transactions on Circuits and Systems for Video Technology , 2023
2023
Later among the works it cites.
J. Liu, Y. Chen, B. Ni, and Z. Yu, “Joint global and dynamic pseudo labeling for semi-supervised point cloud sequence segmentation,” IEEE Transactions on Circuits and Systems for Video Technology , 2023
2023
Later among the works it cites.
P. Mu, G. Wu, J. Liu, Y. Zhang, X. Fan, and R. Liu, “Learning to search a lightweight generalized network for medical image fusion,” IEEE Transactions on Circuits and Systems for Video Technology , 2023
2023
Later among the works it cites.
A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023
Original
2023
Later among the works it cites.
T. Zhou, Y. Zhou, G. Li, G. Chen, and J. Shen, “Uncertainty-aware hierarchical aggregation network for medical image segmentation,” IEEE Transactions on Circuits and Systems for Video Technology , 2024
2024
Closest in time.
L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: Efficient visual representation learning with bidirectional state space model,” arXiv preprint arXiv:2401.09417 , 2024
Original
2024
Closest in time.
Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang, Q. Ye, and Y. Liu, “Vmamba: Visual state space model,” arXiv preprint arXiv:2401.10166 , 2024
Original
2024
Closest in time.
J. Ma, F. Li, and B. Wang, “U-mamba: Enhancing long-range dependency for biomedical image segmentation,” arXiv preprint arXiv:2401.04722 , 2024
Original
2024
Closest in time.
Z. Xing, T. Ye, Y. Yang, G. Liu, and L. Zhu, “Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation,” arXiv preprint arXiv:2401.13560 , 2024
Original
2024
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…