Fetching the paper…

Reading the bibliography…

Evaluating Knowledge Transfer in Neural Network for Medical Images · Around

2020

Evaluating Knowledge Transfer in Neural Network for Medical Images

Akbarian, Sina, Seyyed-Kalantari, Laleh, Khalvati, Farzad et al.

Original
PDF
Post on X

Understand

Deep learning and knowledge transfer techniques have permeated the field of medical imaging and are considered as key approaches for revolutionizing diagnostic imaging practices.

However, there are still challenges for the successful integration of deep learning into medical imaging tasks due to a lack of large annotated imaging data.
To address this issue, we propose a teacher-student learning framework to transfer knowledge from a carefully pre-trained convolutional neural network (CNN) teacher to a student CNN.
In this study, we explore the performance of knowledge transfer in the medical imaging setting.

Built on

J. Irvin, P. Rajpurkar, M. Ko, Y. Yu, S. Ciurea-Ilcus, C. Chute, H. Marklund, B. Haghgoo, R. Ball, K. Shpanskaya, J. Seekins, D. A. Mong, S. S. Halabi, J. K. Sandberg, R. Jones, D. B. Larson, C. P. Langlotz, B. N. Patel, M. P. Lungren, and A. Y. Ng, “CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison,” arXiv:1901.07031 [cs, eess] , Jan. 2019, arXiv: 1901.07031
Original
1901
Earlier work this paper cites.
A. E. W. Johnson, T. J. Pollard, S. J. Berkowitz, N. R. Greenbaum, M. P. Lungren, C.-y. Deng, R. G. Mark, and S. Horng, “MIMIC-CXR: A large publicly available database of labeled chest radiographs,” arXiv:1901.07042 [cs, eess] , Jan. 2019, arXiv: 1901.07042. [Online]. Available: http://arxiv.org/abs/1901.07042
Original
1901
Earlier work this paper cites.
A. Bustos, A. Pertusa, J.-M. Salinas, and M. de la Iglesia-Vayá, “PadChest: A large chest x-ray image dataset with multi-label annotated reports,” arXiv:1901.07441 [cs, eess] , Jan. 2019, arXiv: 1901.07441. [Online]. Available: http://arxiv.org/abs/1901.07441
Original
1901
Earlier work this paper cites.
G. Gardner, D. Keating, T. H. Williamson, and A. T. Elliott, “Automatic detection of diabetic retinopathy using an artificial neural network: a screening tool.” British journal of Ophthalmology , vol. 80, no. 11, pp. 940–944, 1996
1996
Earlier work this paper cites.
C. Buciluǎ, R. Caruana, and A. Niculescu-Mizil, “Model compression,” in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining , 2006, pp. 535–541
2006
Earlier work this paper cites.
S. Gutstein, O. Fuentes, and E. Freudenthal, “Knowledge transfer in deep convolutional neural nets,” International Journal on Artificial Intelligence Tools , vol. 17, no. 03, pp. 555–567, 2008
2008
Earlier work this paper cites.
J. Nayak, P. S. Bhat, R. Acharya, C. M. Lim, and M. Kagathi, “Automated identification of diabetic retinopathy stages using digital fundus images,” Journal of medical systems , vol. 32, no. 2, pp. 107–115, 2008
2008
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in CVPR09 , 2009
2009
Earlier work this paper cites.
J. Cuadros and G. Bresnick, “Eyepacs: an adaptable telemedicine system for diabetic retinopathy screening,” Journal of diabetes science and technology , vol. 3, no. 3, pp. 509–516, May 2009. [Online]. Available: https://pubmed.ncbi.nlm.nih.gov/20144289
2009
Earlier work this paper cites.
S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on Knowledge and Data Engineering , vol. 22, no. 10, pp. 1345–1359, 2010
2010
Earlier work this paper cites.
A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550 , 2014
Original
2014
Earlier work this paper cites.
S. Liu, S. Liu, W. Cai, S. Pujol, R. Kikinis, and D. Feng, “Early diagnosis of alzheimer’s disease with deep learning,” in 2014 IEEE 11th international symposium on biomedical imaging (ISBI) . IEEE, 2014, pp. 1015–1018
2014
Earlier work this paper cites.
G. Torres-Mejía, R. A. Smith, M. d. l. L. Carranza-Flores, A. Bogart, L. Martínez-Matsushita, D. L. Miglioretti, K. Kerlikowske, C. Ortega-Olvera, E. Montemayor-Varela, A. Angeles-Llerenas, S. Bautista-Arredondo, G. Sánchez-González, O. G. Martínez-Montañez, S. R. Uscanga-Sánchez, E. Lazcano-Ponce, and M. Hernández-Ávila, “Radiographers supporting radiologists in the interpretation of screening mammography: a viable strategy to meet the shortage in the number of radiologists,” BMC Cancer , vol. 15, no. 1, p. 410, May 2015. [Online]. Available: https://doi.org/10.1186/s12885-015-1399-2
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015
Original
2015
Earlier work this paper cites.
A. Nishie, D. Kakihara, T. Nojo, K. Nakamura, S. Kuribayashi, M. Kadoya, K. Ohtomo, K. Sugimura, and H. Honda, “Current radiologist workload and the shortages in Japan: how many full-time radiologists are required?” Japanese Journal of Radiology , vol. 33, no. 5, pp. 266–272, May 2015. [Online]. Available: https://doi.org/10.1007/s11604-015-0413-6
2015
Earlier work this paper cites.
D. Rosman, J. J. Nshizirungu, E. Rudakemwa, C. Moshi, J. d. D. Tuyisenge, E. Uwimana, and L. Kalisa, “Imaging in the Land of 1000 Hills: Rwanda Radiology Country Report,” Journal of Global Radiology , vol. 1, no. 1, Mar. 2015. [Online]. Available: https://escholarship.umassmed.edu/jgr/vol1/iss1/5
2015
Earlier work this paper cites.
F. S Ali, S. G Harrington, S. B Kenned, and S. Hussain, “Diagnostic radiology in liberia: a country report,” Journal of Global Radiology , vol. 1(2), no. 6, 2015
2015
Earlier work this paper cites.
S. Zagoruyko and N. Komodakis, “Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer,” CoRR , vol. abs/1612.03928, 2016. [Online]. Available: http://arxiv.org/abs/1612.03928
Original
2016
Earlier work this paper cites.
H. Pratt, F. Coenen, D. M. Broadbent, S. P. Harding, and Y. Zheng, “Convolutional neural networks for diabetic retinopathy,” Procedia Computer Science , vol. 90, pp. 200–205, 2016
2016
Earlier work this paper cites.

Similar

V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan, K. Widner, T. Madams, J. Cuadros, R. Kim, R. Raman, P. C. Nelson, J. L. Mega, and D. R. Webster, “Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs,” JAMA , vol. 316, no. 22, pp. 2402–2410, 12 2016. [Online]. Available: https://doi.org/10.1001/jama.2016.17216
2016
Cited alongside, same era.
A. Rimmer, “Radiologist shortage leaves patient care at risk, warns royal college,” BMJ (Clinical research ed.) , vol. 359, p. j4683, 2017
2017
Cited alongside, same era.
S. Bastawrous and B. Carney, “Improving Patient Safety: Avoiding Unread Imaging Exams in the National VA Enterprise Electronic Health Record,” Journal of Digital Imaging , vol. 30, no. 3, 2017
2017
Cited alongside, same era.
P. Lakhani and B. Sundaram, “Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks,” Radiology , vol. 284, no. 2, pp. 574–582, Aug. 2017. [Online]. Available: http://pubs.rsna.org/doi/10.1148/radiol.2017162326

Then

Q. Guan, Y. Huang, Z. Zhong, Z. Zheng, L. Zheng, and Y. Yang, “Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification,” arXiv:1801.09927 [cs] , Jan. 2018, arXiv: 1801.09927. [Online]. Available: http://arxiv.org/abs/1801.09927
Original
2018
Later among the works it cites.
A. B. Rosenkrantz, W. Wang, D. R. Hughes, and R. Duszak, “A County-Level Analysis of the US Radiologist Workforce: Physician Supply and Subspecialty Characteristics,” Journal of the American College of Radiology: JACR , vol. 15, no. 4, pp. 601–606, Apr. 2018
2018
Later among the works it cites.
J. R. Zech, M. A. Badgeley, M. Liu, A. B. Costa, J. J. Titano, and E. K. Oermann, “Confounding variables can degrade generalization performance of radiological deep learning models,” PLOS Medicine , vol. 15, no. 11, p. e1002683, Nov. 2018, arXiv: 1807.00431. [Online]. Available: http://arxiv.org/abs/1807.00431
Original
2018
Later among the works it cites.
Z. Li, C. Wang, M. Han, Y. Xue, W. Wei, L.-J. Li, and L. Fei-Fei, “Thoracic disease identification and localization with limited supervision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 8290–8299

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2017

Cited alongside, same era.

P. Rajpurkar, J. Irvin, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, A. Bagul, C. Langlotz, K. Shpanskaya, M. P. Lungren, and A. Y. Ng, “CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning,” arXiv:1711.05225 [cs, stat] , Nov. 2017, arXiv: 1711.05225. [Online]. Available: http://arxiv.org/abs/1711.05225

2017

Cited alongside, same era.

J. Yim, D. Joo, J. Bae, and J. Kim, “A gift from knowledge distillation: fast optimization, network minimization and transfer learning,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017

2017

Cited alongside, same era.

A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer with deep neural networks,” Nature , vol. 542, no. 7639, pp. 115–118, Feb. 2017. [Online]. Available: https://www.nature.com/articles/nature21056

2017

Cited alongside, same era.

N. C. Codella, Q.-B. Nguyen, S. Pankanti, D. A. Gutman, B. Helba, A. C. Halpern, and J. R. Smith, “Deep learning ensembles for melanoma recognition in dermoscopy images,” IBM Journal of Research and Development , vol. 61, no. 4/5, pp. 5–1, 2017

2017

Cited alongside, same era.

L. Yao, E. Poblenz, D. Dagunts, B. Covington, D. Bernard, and K. Lyman, “Learning to diagnose from scratch by exploiting dependencies among labels,” arXiv:1710.10501 [cs] , Oct. 2017, arXiv: 1710.10501. [Online]. Available: http://arxiv.org/abs/1710.10501

2017

Cited alongside, same era.

X. Li, T. Pang, B. Xiong, W. Liu, P. Liang, and T. Wang, “Convolutional neural networks based transfer learning for diabetic retinopathy fundus image classification,” in 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI) . IEEE, 2017, pp. 1–11

2017

Cited alongside, same era.

S. Masood, T. Luthra, H. Sundriyal, and M. Ahmed, “Identification of diabetic retinopathy in eye images using transfer learning,” in 2017 International Conference on Computing, Communication and Automation (ICCCA) . IEEE, 2017, pp. 1183–1187

2017

Cited alongside, same era.

C. Rubin, “Clinical radiology UK workforce census 2017 report,” The Royal college of radiologists , p. 40, 2017

2017

Cited alongside, same era.

2018

Later among the works it cites.

A. S. Lundervold and A. Lundervold, “An overview of deep learning in medical imaging focusing on mri,” Zeitschrift für Medizinische Physik , vol. 29, no. 2, pp. 102–127, 2019

2019

Later among the works it cites.

Y. Jang, H. Lee, S. J. Hwang, and J. Shin, “Learning what and where to transfer,” arXiv preprint arXiv:1905.05901 , 2019

2019

Later among the works it cites.

F. Knoll, K. Hammernik, E. Kobler, T. Pock, M. P. Recht, and D. K. Sodickson, “Assessment of the generalization of learned image reconstruction and the potential for transfer learning,” Magnetic resonance in medicine , vol. 81, no. 1, pp. 116–128, 2019

2019

Later among the works it cites.

D. Hendrycks, K. Lee, and M. Mazeika, “Using pre-training can improve model robustness and uncertainty,” 2019

2019

Later among the works it cites.

M. Raghu, C. Zhang, J. Kleinberg, and S. Bengio, “Transfusion: Understanding Transfer Learning for Medical Imaging,” in Proceedings of the 33rd International Conference on Neural Information Processing Systems , 2019

2019

Later among the works it cites.

Y. Jang, H. Lee, S. J. Hwang, and J. Shin, “Learning what and where to transfer,” in ICML , 2019

2019

Later among the works it cites.

N. Garin, C. Marti, M. Scheffler, J. Stirnemann, and V. Prendki, “Computed tomography scan contribution to the diagnosis of community-acquired pneumonia,” Current opinion in pulmonary medicine , vol. 25, no. 3, p. 242, 2019

2019

Later among the works it cites.

L. Seyyed-Kalantari, G. Liu, M. McDermott, and G. Marzyeh, “Chexclusion: Fairness gaps in deep chest x-ray classifiers,” arXiv preprint arXiv:2003.00827 , 2020

2020

Closest in time.

S. Akbarian, N. Montazeri Ghahjaverestan, A. Yadollahi, and B. Taati, “Distinguishing obstructive versus central apneas in infrared video of sleep using deep learning: Validation study,” J Med Internet Res , vol. 22, no. 5, p. e17252, May 2020. [Online]. Available: http://www.jmir.org/2020/5/e17252/

2020

Closest in time.

J. P. Cohen, L. Dao, P. Morrison, K. Roth, Y. Bengio, B. Shen, A. Abbasi, M. Hoshmand-Kochi, M. Ghassemi, H. Li, and T. Q. Duong, “Predicting covid-19 pneumonia severity on chest x-ray with deep learning,” 2020

2020

Closest in time.

Y. Tian, Y. Wang, D. Krishnan, J. B. Tenenbaum, and P. Isola, “Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?” arXiv e-prints , p. arXiv:2003.11539, Mar. 2020

Original

2020

Closest in time.

M. Iglesia la de Vayá, J. M. Saborit, J. A. Montell, A. Pertusa, A. Bustos, M. Cazorla, J. Galant, X. Barber, D. Orozco-Beltrán, F. García-García, M. Caparrós, G. González, and J. María Salinas, “BIMCV COVID-19+: a large annotated dataset of RX and CT images from COVID-19 patients,” arXiv e-prints , p. arXiv:2006.01174, Jun. 2020

2020

Closest in time.

J. P. Cohen, P. Morrison, L. Dao, K. Roth, T. Q. Duong, and M. Ghassemi, “COVID-19 Image Data Collection: Prospective Predictions Are the Future,” arXiv e-prints , p. arXiv:2006.11988, Jun. 2020

2020

Closest in time.

X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “ChestX-ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases,” in Computer Vision and Pattern Recognition (CVPR) 2017 . IEEE, 2017, pp. 2097–2106. [Online]. Available: http://openaccess.thecvf.com/content_cvpr_2017/html/Wang_ChestX-ray8_Hospital-Scale_Chest_CVPR_2017_paper.html

2097

Closest in time.