Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis

A new approach to linear filtering and prediction problems

Rudolph Emil Kalman · 1960

Earlier work this paper cites.

Long short-term memory

Sepp Hochreiter and Jürgen Schmidhuber · 1997

Earlier work this paper cites.

Imagenet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009

Earlier work this paper cites.

Learning phrase representations using rnn encoder-decoder for statistical machine translation

Original

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio · 2014

Earlier work this paper cites.

A novel method for implementation of frameless stereoeeg in epilepsy surgery

Mark Nowell, Roman Rodionov, Beate Diehl, Tim Wehner, Gergely Zombori, Jane Kinghorn, Sebastien Ourselin, John Duncan, Anna Miserocchi, and Andrew McEvoy · 2014

Earlier work this paper cites.

U-net: Convolutional networks for biomedical image segmentation

Olaf Ronneberger, Philipp Fischer, and Thomas Brox · 2015

Earlier work this paper cites.

Mimic-iii, a freely accessible critical care database

Alistair E W Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark · 2016

Earlier work this paper cites.

Fully-convolutional siamese networks for object tracking

Luca Bertinetto, Jack Valmadre, Joao F. Henriques, Andrea Vedaldi, and Philip H. S. Torr · 2016

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017

Earlier work this paper cites.

Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved?

Olivier Bernard, Alain Lalande, Clement Zotti, Frederick Cervenansky, Xin Yang, Pheng-Ann Heng, Irem Cetin, Karim Lekadir, Oscar Camara, Miguel Angel Gonzalez Ballester, et al · 2018

Earlier work this paper cites.

An image is worth 16x16 words: Transformers for image recognition at scale

Original

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020

Earlier work this paper cites.

Hippo: Recurrent memory with optimal polynomial projections, 2020

Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Re · 2020

Earlier work this paper cites.

Inferring super-resolution depth from a moving light-source enhanced rgb-d sensor: a variational approach

Lu Sang, Bjoern Haefner, and Daniel Cremers · 2020

Earlier work this paper cites.

Cnn variants for computer vision: History, architecture, application, challenges and future scope

Dulari Bhatt, Chirag Patel, Hardik Talsania, Jigar Patel, Rasmika Vaghela, Sharnil Pandya, Kirit Modi, and Hemant Ghayvat · 2021

Earlier work this paper cites.

Transunet: Transformers make strong encoders for medical image segmentation

Original

Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou · 2021

Earlier work this paper cites.

Brain tumor detection analysis using cnn: a review

Sunil Kumar, Renu Dhir, and Nisha Chaurasia · 2021

Earlier work this paper cites.

Combining recurrent, convolutional, and continuous-time models with linear state-space layers, 2021

Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré · 2021

Earlier work this paper cites.

Training data-efficient image transformers & distillation through attention

Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou · 2021

Earlier work this paper cites.

Medical image segmentation with limited supervision: a review of deep network models

Jialin Peng and Ye Wang · 2021

Earlier work this paper cites.

Emerging properties in self-supervised vision transformers

Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin · 2021

Earlier work this paper cites.

Convolutional neural networks in medical image understanding: a survey

DR Sarvamangala and Raghavendra V Kulkarni · 2022

Earlier work this paper cites.

Transformers in vision: A survey

Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah · 2022

Earlier work this paper cites.

Building efficient cnn architectures for histopathology images analysis: A case-study in tumor-infiltrating lymphocytes classification

André LS Meirelles, Tahsin Kurc, Jun Kong, Renato Ferreira, Joel H Saltz, and George Teodoro · 2022

Earlier work this paper cites.

Modeling long-range dependencies for weakly supervised disease classification and localization on chest x-ray

Fangyun Li, Lingxiao Zhou, Yunpeng Wang, Chuan Chen, Shuyi Yang, Fei Shan, and Lei Liu · 2022

Earlier work this paper cites.

Efficient transformers: A survey

Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler · 2022

Earlier work this paper cites.

Billion-scale pretraining with vision transformers for multi-task visual representations

Josh Beal, Hao-Yu Wu, Dong Huk Park, Andrew Zhai, and Dmitry Kislyuk · 2022

Earlier work this paper cites.

Efficiently modeling long sequences with structured state spaces, 2022

Albert Gu, Karan Goel, and Christopher Ré · 2022

Earlier work this paper cites.

On the parameterization and initialization of diagonal state space models, 2022

Albert Gu, Ankit Gupta, Karan Goel, and Christopher Ré · 2022

Earlier work this paper cites.

Diagonal state spaces are as effective as structured state spaces, 2022

Ankit Gupta, Albert Gu, and Jonathan Berant · 2022

Earlier work this paper cites.

What makes convolutional models great on long sequence modeling?, 2022

Yuhong Li, Tianle Cai, Yi Zhang, Deming Chen, and Debadeepta Dey · 2022

Earlier work this paper cites.

Long range language modeling via gated state spaces, 2022

Harsh Mehta, Ankit Gupta, Ashok Cutkosky, and Behnam Neyshabur · 2022

Earlier work this paper cites.

Medical image segmentation on mri images with missing modalities: A review

Original

Reza Azad, Nika Khosravi, Mohammad Dehghanmanshadi, Julien Cohen-Adad, and Dorit Merhof · 2022

Earlier work this paper cites.

Deep learning for medical image analysis

S Kevin Zhou, Hayit Greenspan, and Dinggang Shen · 2023

Earlier work this paper cites.

A comprehensive review of deep neural networks for medical image processing: Recent developments and future opportunities

Pawan Kumar Mall, Pradeep Kumar Singh, Swapnita Srivastav, Vipul Narayan, Marcin Paprzycki, Tatiana Jaworska, and Maria Ganzha · 2023

Earlier work this paper cites.

Advances in medical image analysis with vision transformers: a comprehensive review

Reza Azad, Amirhossein Kazerouni, Moein Heidari, Ehsan Khodapanah Aghdam, Amirali Molaei, Yiwei Jia, Abin Jose, Rijo Roy, and Dorit Merhof · 2023

Earlier work this paper cites.

A survey of visual transformers

Yang Liu, Yao Zhang, Yixin Wang, Feng Hou, Jin Yuan, Jiang Tian, Yang Zhang, Zhongchao Shi, Jianping Fan, and Zhiqiang He · 2023

Earlier work this paper cites.

A survey on deep learning applied to medical images: from simple artificial neural networks to generative models

Pedro Celard, Eva Lorenzo Iglesias, José Manuel Sorribes-Fdez, Rubén Romero, A Seara Vieira, and Lourdes Borrajo · 2023

Earlier work this paper cites.

Fusion of u-net and cnn model for segmentation and classification of skin lesion from dermoscopy images

Vatsala Anand, Sheifali Gupta, Deepika Koundal, and Karamjeet Singh · 2023

Earlier work this paper cites.

Brain tumor segmentation and classification using hybrid deep cnn with lunetclassifier

T Balamurugan and E Gnanamanoharan · 2023

Earlier work this paper cites.

Lightweight deep cnn-based models for early detection of covid-19 patients from chest x-ray images

Haval I Hussein, Abdulhakeem O Mohammed, Masoud M Hassan, and Ramadhan J Mstafa · 2023

Earlier work this paper cites.

Gil-cnn: A novel multipath features for covid-19 detection using ct-scan images

N Jagan Mohan and DN Kiran Pandiri · 2023

Earlier work this paper cites.

Multiple brain tumor classification with dense cnn architecture using brain mri images

Osman Özkaraca, Okan İhsan Bağrıaçık, Hüseyin Gürüler, Faheem Khan, Jamil Hussain, Jawad Khan, and Umm e Laila · 2023

Earlier work this paper cites.

High accuracy hybrid cnn classifiers for breast cancer detection using mammogram and ultrasound datasets

Adyasha Sahu, Pradeep Kumar Das, and Sukadev Meher · 2023

Earlier work this paper cites.

Transformers in medical imaging: A survey

Fahad Shamshad, Salman Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, and Huazhu Fu · 2023

Earlier work this paper cites.

Transformers in medical image analysis

Kelei He, Chen Gan, Zhuoyuan Li, Islem Rekik, Zihao Yin, Wen Ji, Yang Gao, Qian Wang, Junfeng Zhang, and Dinggang Shen · 2023

Earlier work this paper cites.

Hiformer: Hierarchical multi-scale representations using transformers for medical image segmentation

Moein Heidari, Amirhossein Kazerouni, Milad Soltany, Reza Azad, Ehsan Khodapanah Aghdam, Julien Cohen-Adad, and Dorit Merhof · 2023

Earlier work this paper cites.

Mamba: Linear-time sequence modeling with selective state spaces, 2023

Albert Gu and Tri Dao · 2023

Earlier work this paper cites.

Hungry hungry hippos: Towards language modeling with state space models, 2023

Daniel Y. Fu, Tri Dao, Khaled K. Saab, Armin W. Thomas, Atri Rudra, and Christopher Ré · 2023

Earlier work this paper cites.

Simplified state space layers for sequence modeling, 2023

Jimmy T. H. Smith, Andrew Warrington, and Scott W. Linderman · 2023

Earlier work this paper cites.

Simple hardware-efficient long convolutions for sequence modeling, 2023

Daniel Y. Fu, Elliot L. Epstein, Eric Nguyen, Armin W. Thomas, Michael Zhang, Tri Dao, Atri Rudra, and Christopher Ré · 2023

Cited alongside, same era.

Hyena hierarchy: Towards larger convolutional language models, 2023

Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y. Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Stefano Ermon, and Christopher Ré · 2023

Cited alongside, same era.

Rwkv: Reinventing rnns for the transformer era, 2023

Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Jiaju Lin, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Bolun Wang, Johan S. Wind, Stanislaw Wozniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Qinghua Zhou, Jian Zhu, and Rui-Jie Zhu · 2023

Cited alongside, same era.

Mega: Moving average equipped gated attention, 2023

Xuezhe Ma, Chunting Zhou, Xiang Kong, Junxian He, Liangke Gui, Graham Neubig, Jonathan May, and Luke Zettlemoyer · 2023

Cited alongside, same era.

Retentive network: A successor to transformer for large language models, 2023

Zigma: Zigzag mamba diffusion model

Original

Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Fischer, and Bjorn Ommer · 2024

Closest in time.

Mamba-nd: Selective state space modeling for multi-dimensional data

Original

Shufan Li, Harkanwar Singh, and Aditya Grover · 2024

Closest in time.

Localmamba: Visual state space model with windowed selective scan

Original

Tao Huang, Xiaohuan Pei, Shan You, Fei Wang, Chen Qian, and Chang Xu · 2024

Closest in time.

Plainmamba: Improving non-hierarchical mamba in visual recognition

Original

Chenhongyi Yang, Zehui Chen, Miguel Espinosa, Linus Ericsson, Zhenyu Wang, Jiaming Liu, and Elliot J Crowley · 2024

Closest in time.

Vivim: a video vision mamba for medical video object segmentation, 2024