Fetching the paper…

Reading the bibliography…

2021

TransMed: Transformers Advance Multi-modal Medical Image Classification

Dai, Yin, Gao, Yifan

Original
PDF
Post on X

Understand

Over the past decade, convolutional neural networks (CNN) have shown very competitive performance in medical image analysis tasks, such as disease classification, tumor segmentation, and lesion detection.

CNN has great advantages in extracting local features of images.
However, due to the locality of convolution operation, it can not deal with long-range relationships well.
Recently, transformers have been applied to computer vision and achieved remarkable success in large-scale datasets.

TransMed: Transformers Advance Multi-modal Medical Image Classification · Around

Built on

A. Paszke et al., “ PyTorch: An Imperative Style, High-Performance Deep Learning Library,” ArXiv , [Online]. Available: http://arxiv.org/abs/1912.01703
Original
1912
Earlier work this paper cites.
Otsu N., “A threshold selection method from gray-level histograms,” in IEEE Trans. Systems, Man, and Cybernetics. , vol. 9, no. 1, pp. 62-66, 1979
1979
Earlier work this paper cites.
V. Q. Joe and P. L. Westesson., “Tumors of the parotid gland: MR imaging characteristics of various histologic types,” in Am. J. Roentgenol. , vol. 163, no. 2, pp. 433–438, Aug. 1994
1994
Earlier work this paper cites.
F. Pérez-García, R. Sparks, and S. Ourselin, “TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning,” in Arxiv , [Online]. Available: http://arxiv.org/abs/2003.04696
Original
2003
Earlier work this paper cites.
R. A. Lima et al., “ Clinical Prognostic Factors in Malignant Parotid Gland Tumors,” in Otolaryngol. Neck Surg. , vol. 133, no. 5, pp. 702–708, Nov. 2005
2005
Earlier work this paper cites.
J. Deng et al., “ ImageNet: A large-scale hierarchical image database,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Jun. 2009, pp. 248–255, doi: 10.1109/CVPR.2009.5206848
2009
Earlier work this paper cites.
A. Dosovitskiy et al., “ An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” ArXiv , [Online]. Available: http://arxiv.org/abs/2010.11929
Original
2010
Earlier work this paper cites.
X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable DETR: Deformable Transformers for End-to-End Object Detection,” in Arxiv , [Online]. Available: http://arxiv.org/abs/2010.04159
Original
2010
Earlier work this paper cites.
S. Zheng et al., “ Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers,” ArXiv , [Online]. Available: http://arxiv.org/abs/2012.15840
Original
2012
Earlier work this paper cites.
H. Touvron et al., “ Training data-efficient image transformers & \& distillation through attention,” ArXiv , [Online]. Available: http://arxiv.org/abs/2012.12877
Original
2012
Earlier work this paper cites.

Similar

D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning Spatiotemporal Features with 3D Convolutional Networks,” in Proc. IEEE Inter. Conf. Comput. Vis. (ICCV) , Santiago, Chile, Dec. 2015, pp. 4489–4497
2015
Cited alongside, same era.
K. Kamnitsas et al., “ Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation,” in Med. Image Anal. , vol. 36, pp. 61–78, Feb. 2017, doi: 10.1016/j.media.2016.10.004
2016
Cited alongside, same era.
D. Nie, L. Wang, Y. Gao, and D. Shen, “Fully convolutional networks for multi-modality isointense infant brain image segmentation,” in 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) , Apr. 2016, pp. 1342–1345, doi: 10.1109/ISBI.2016.7493515
2016
Cited alongside, same era.
A. A. A. Setio et al., “ MMFNet: A multi-modality MRI fusion network for segmentation of nasopharyngeal carcinoma,” in IEEE Trans. Med. Imaging. , vol. 35, no. 5, pp. 1160–1169, May 2016, doi: 10.1109/TMI.2016.2536809
2016

Then

Z. Guo, X. Li, H. Huang, N. Guo, and Q. Li, “Medical image segmentation based on multi-modal convolutional neural network: Study on image fusion schemes,” in 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI) , Apr. 2018, pp. 903–907, doi: 10.1109/ISBI.2018.8363717
2018
Later among the works it cites.
X. Wang, R. Girshick, A. Gupta, and K. He, “Non-Local Neural Networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2018, pp. 7794-7803
2018
Later among the works it cites.
T. Zhou, S. Ruan, and S. Canu, “A review: Deep learning for medical image segmentation using multi-modality fusion,” in Array. , vol. 3–4, p. 100004, Sep. 2019, doi: 10.1016/j.array.2019.100004
2019
Later among the works it cites.
A. Myronenko et al., “ 3D MRI Brain Tumor Segmentation Using Autoencoder Regularization,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. Workshop(MICCAI) , 2019, vol. 11384, pp. 311–320, doi: 10.1007/978-3-030-11726-9
2019

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Las Vegas, NV, USA, Jun. 2016, pp. 770–778

2016

Cited alongside, same era.

Vaswani A et al., “ Attention is All you Need,” in Proc. Conf. Neural Inf. Process. Syst. (NeurIPS) , 2017, pp 1-12

2017

Cited alongside, same era.

K.-L. Tseng, Y.-L. Lin, W. Hsu, and C.-Y. Huang, “Joint Sequence Learning and Cross-Modality Convolution for 3D Biomedical Segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Jul. 2017, pp. 3739–3746, doi: 10.1109/CVPR.2017.398

2017

Cited alongside, same era.

Z. Qiu, T. Yao, and T. Mei, “Learning Spatio-Temporal Representation With Pseudo-3D Residual Networks,” in Proc. IEEE Inter. Conf. Comput. Vis. (ICCV) , 2017, pp. 5533-5541

2017

Cited alongside, same era.

F. Isensee, P. Kickingereder, W. Wick, M. Bendszus, and K. H. Maier-Hein, “Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. Workshop(MICCAI) , 2018, pp. 287–297, doi: 10.1007/978-3-319-75238-9

2018

Cited alongside, same era.

J. Dolz, K. Gopinath, J. Yuan, H. Lombaert, C. Desrosiers, and I. Ben Ayed, “HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation,” in IEEE Trans. Med. Imaging , vol. 38, no. 5, pp. 1116–1126, May 2019, doi: 10.1109/TMI.2018.2878669

2018

Cited alongside, same era.

D. Hendrycks and K. Gimpel, “Gaussian Error Linear Units (GELUs),” in Arxiv , [Online]. Available: http://arxiv.org/abs/1606.08415

Cited in the paper.

Later among the works it cites.

Y. Shachor, H. Greenspan, and J. Goldberger, “A mixture of views network with applications to multi-view medical imaging,” in Neurocomputing , vol. 374, pp. 1–9, Jan. 2020, doi: 10.1016/j.neucom.2019.09.027

2019

Later among the works it cites.

Carion et al., “ End-to-end object detection with transformers,” in Proc. Eur. Conf. Comput. Vis. (ECCV) , 2020, pp. 213-229

2020

Later among the works it cites.

H. Chen et al., “ MMFNet: A multi-modality MRI fusion network for segmentation of nasopharyngeal carcinoma,” in Neurocomputing. , 2020, vol. 394, pp. 27-40, doi: https://doi.org/10.1016/j.neucom.2020.02.002

2020

Later among the works it cites.

Chen M et al., “ Generative pretraining from pixels,” in Int. Conf. on Mach. Learn. (ICML) , 2020, pp. 1691-1703

2020

Later among the works it cites.

J.-S. Jiang et al., “ Added value of susceptibility-weighted imaging to diffusion-weighted imaging in the characterization of parotid gland tumors,” in Eur. Arch. Otorhinolaryngol. , ol. 277, no. 10, pp. 2839–2846, Oct. 2020

2020

Later among the works it cites.