2017

DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation

Wang, Guotai, Zuluaga, Maria A., Li, Wenqi et al.

Understand

Accurate medical image segmentation is essential for diagnosis, surgical planning and many other applications.

Convolutional Neural Networks (CNNs) have become the state-of-the-art automatic segmentation methods.
However, fully automatic results may still need to be refined to become accurate and robust enough for clinical use.
We propose a deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy.

Built on

C. Xu and J. L. Prince, “Snakes, Shapes, and Gradient Vector Flow,” TIP , vol. 7, no. 3, pp. 359–369, 1998
1998
Earlier work this paper cites.
Y. Boykov and M.-P. Jolly, “Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images,” in ICCV , 2001, pp. 105–112
2001
Earlier work this paper cites.
C. Rother, V. Kolmogorov, and A. Blake, “”GrabCut”: Interactive Foreground Extraction Using Iterated Graph Cuts,” ACM Trans. on Graphics , vol. 23, no. 3, pp. 309–314, 2004
2004
Earlier work this paper cites.
J. E. Cates, A. E. Lefohn, and R. T. Whitaker, “GIST: An interactive, GPU-based level set segmentation tool for 3D medical images,” Medical Image Analysis , vol. 8, no. 3, pp. 217–231, 2004
2004
Earlier work this paper cites.
Y. Boykov and V. Kolmogorov, “An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision,” PAMI , vol. 26, no. 9, pp. 1124–1137, 2004
2004
Earlier work this paper cites.
V. Kolmogorov and R. Zabih, “What Energy Functions Can Be Minimized via Graph Cuts?” PAMI , vol. 26, no. 2, pp. 147–159, 2004
2004
Earlier work this paper cites.
L. Grady, “Random walks for image segmentation,” PAMI , vol. 28, no. 11, pp. 1768–1783, 2006
2006
Earlier work this paper cites.
P. A. Yushkevich, J. Piven, H. C. Hazlett, R. G. Smith, S. Ho, J. C. Gee, and G. Gerig, “User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability,” NeuroImage , vol. 31, no. 3, pp. 1116–1128, 2006
2006
Earlier work this paper cites.
C. J. Armstrong, B. L. Price, and W. A. Barrett, “Interactive segmentation of image volumes with Live Surface,” Computers and Graphics (Pergamon) , vol. 31, no. 2, pp. 212–229, 2007
2007
Earlier work this paper cites.
X. Bai and G. Sapiro, “A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting,” in ICCV , oct 2007
2007
Earlier work this paper cites.
A. Criminisi, T. Sharp, and A. Blake, “GeoS: Geodesic Image Segmentation,” in ECCV , 2008
2008
Earlier work this paper cites.
M. Szummer, P. Kohli, and D. Hoiem, “Learning CRFs using graph cuts,” in ECCV , 2008
2008
Earlier work this paper cites.
N. Sharma and L. M. Aggarwal, “Automated medical image segmentation techniques.” Journal of medical physics , vol. 35, no. 1, pp. 3–14, 2010
2010
Earlier work this paper cites.
J. Yuan, E. Bae, and X. C. Tai, “A study on continuous max-flow and min-cut approaches,” in CVPR , 2010
2010
Earlier work this paper cites.
N. Payet and S. Todorovic, “(RF)ˆ2 – Random Forest Random Field,” NIPS , vol. 1, pp. 1885–1893, 2010
2010
Earlier work this paper cites.
A. Adams, J. Baek, and M. A. Davis, “Fast high-dimensional filtering using the permutohedral lattice,” Computer Graphics Forum , vol. 29, no. 2, pp. 753–762, 2010
2010
Earlier work this paper cites.
J. A. Deprest, A. W. Flake, E. Gratacos, Y. Ville, K. Hecher, K. Nicolaides, M. P. Johnson, F. I. Luks, N. S. Adzick, and M. R. Harrison, “The Making of Fetal Surgery,” Prenatal Diagnosis , vol. 30, no. 7, pp. 653–667, 2010
2010
Earlier work this paper cites.
P. Krähenbühl and V. Koltun, “Efficient inference in fully connected CRFs with gaussian edge potentials,” in NIPS , 2011
2011
Earlier work this paper cites.
D. Han, J. Bayouth, Q. Song, A. Taurani, M. Sonka, J. Buatti, and X. Wu, “Globally optimal tumor segmentation in PET-CT images: A graph-based co-segmentation method,” in IPMI , 2011
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in NIPS , 2012
2012
Earlier work this paper cites.
O. Barinova, R. Shapovalov, S. Sudakov, and A. Velizhev, “Online Random Forest for Interactive Image Segmentation,” in EEML , 2012
2012
Earlier work this paper cites.
F. Zhao and X. Xie, “An Overview of Interactive Medical Image Segmentation,” Annals of the BMVA , vol. 2013, no. 7, pp. 1–22, 2013
2013
Earlier work this paper cites.

Similar

P. Kohli, J. Shotton, and A. Criminisi, “GeoF Geodesic Forests for Learning Coupled Predictors,” in CVPR , 2013
2013
Cited alongside, same era.
J. Domke, “Learning graphical model parameters with approximate marginal inference,” PAMI , vol. 35, no. 10, pp. 2454–2467, 2013
2013
Cited alongside, same era.
P. Krähenbühl and V. Koltun, “Parameter Learning and Convergent Inference for Dense Random Fields,” in ICML , 2013, pp. 1–9
2013
Cited alongside, same era.
B. Wang, W. Liu, M. Prastawa, A. Irimia, P. M. Vespa, J. D. V. Horn, P. T. Fletcher, and G. Gerig, “4D active cut: An interactive tool for pathological anatomy modeling,” in ISBI , 2014
2014
Cited alongside, same era.
R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,” in CVPR , 2014
2014
Cited alongside, same era.
P. Pinheiro and R. Collobert, “Recurrent convolutional neural networks for scene labeling,” in ICML , 2014
2014
Cited alongside, same era.
J. I. Orlando and M. Blaschko, “Learning Fully-Connected CRFs for Blood Vessel Segmentation in Retinal Images,” in MICCAI , 2014
2014
Cited alongside, same era.
F. Liu, C. Shen, and G. Lin, “Deep Convolutional Neural Fields for Depth Estimation from a Single Image,” in CVPR , 2014
2014
Cited alongside, same era.
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, “Caffe: Convolutional Architecture for Fast Feature Embedding,” in ACMICM , 2014
2014
Cited alongside, same era.
K. Keraudren, M. Kuklisova-Murgasova, V. Kyriakopoulou, C. Malamateniou, M. A. Rutherford, B. Kainz, J. V. Hajnal, and D. Rueckert, “Automated Fetal Brain Segmentation from 2D MRI Slices for Motion Correction,” NeuroImage , vol. 101, no. 1 November 2014, pp. 633–643, 2014
2014
Cited alongside, same era.
L. Fidon, W. Li, L. C. Garcia-peraza herrera, J. Ekanayake, N. Kitchen, S. Ourselin, and T. Vercauteren, “Scalable multimodel convolutional networks for brain tumour segmentation,” in MICCAI , 2017
2014
Cited alongside, same era.
J. Long, E. Shelhamer, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” in CVPR , 2015, pp. 3431–3440
2015
Cited alongside, same era.

Then

M. Havaei, A. Davy, D. Warde-Farley, A. Biard, A. Courville, Y. Bengio, C. Pal, P.-M. Jodoin, and H. Larochelle, “Brain Tumor Segmentation with Deep Neural Networks,” Medical Image Analysis , vol. 35, pp. 18–31, 2016
2016
Later among the works it cites.
F. Yu and V. Koltun, “Multi-Scale Context Aggregation By Dilated Convolutions,” in ICLR , 2016
2016
Later among the works it cites.
A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3D U-Net : Learning Dense Volumetric Segmentation from Sparse Annotation,” in MICCAI , 2016
2016
Later among the works it cites.
D. Lin, J. Dai, J. Jia, K. He, and J. Sun, “ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation,” in CVPR , 2016
2016
Later among the works it cites.
M. Rajchl, M. Lee, O. Oktay, K. Kamnitsas, J. Passerat-Palmbach, W. Bai, M. Rutherford, J. Hajnal, B. Kainz, and D. Rueckert, “DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks,” TMI , vol. PP, no. 99, pp. 1–1, 2016
2016
Later among the works it cites.
N. Xu, B. Price, S. Cohen, J. Yang, and T. Huang, “Deep Interactive Object Selection,” in CVPR , 2016, pp. 373–381
2016
Later among the works it cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in CVPR , 2016
2016
Later among the works it cites.
F. Milletari, N. Navab, and S.-A. Ahmadi, “V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation,” in IC3DV , 2016, pp. 565–571
2016
Later among the works it cites.
P. Ondruska, J. Dequaire, D. Z. Wang, and I. Posner, “End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks,” in Robotics: Science and Systems, Workshop on Limits and Potentials of Deep Learning in Robotics , 2016
2016
Later among the works it cites.
J. Dai, K. He, Y. Li, S. Ren, and J. Sun, “Instance-sensitive Fully Convolutional Networks,” in ECCV , 2016
2016
Later among the works it cites.
C. Lea, R. Vidal, A. Reiter, and G. D. Hager, “Temporal Convolutional Networks: A Unified Approach to Action Segmentation,” in ECCV , 2016
2016
Later among the works it cites.
G. Lin, C. Shen, I. Reid, and A. van dan Hengel, “Efficient piecewise training of deep structured models for semantic segmentation,” in CVPR , 2016
2016
Later among the works it cites.
R. Vemulapalli, O. Tuzel, M.-y. Liu, and R. Chellappa, “Gaussian Conditional Random Field Network for Semantic Segmentation,” in CVPR , 2016
2016
Later among the works it cites.
A. Kirillov, S. Zheng, D. Schlesinger, W. Forkel, A. Zelenin, P. Torr, and C. Rother, “Efficient Likelihood Learning of a Generic CNN-CRF Model for Semantic Segmentation,” in ACCV , 2016
2016
Later among the works it cites.
G. Wang, M. A. Zuluaga, R. Pratt, M. Aertsen, T. Doel, M. Klusmann, A. L. David, J. Deprest, T. Vercauteren, and S. Ourselin, “Dynamically Balanced Online Random Forests for Interactive Scribble-Based Segmentation,” in MICCAI , 2016
2016
Later among the works it cites.
M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, X. Zheng, and G. Brain, “TensorFlow: A System for Large-Scale Machine Learning TensorFlow: A system for large-scale machine learning,” in OSDI , 2016, pp. 265–284
2016
Later among the works it cites.
A. Alansary, K. Kamnitsas, A. Davidson, M. Rajchl, C. Malamateniou, M. Rutherford, J. V. Hajnal, B. Glocker, D. Rueckert, and B. Kainz, “Fast Fully Automatic Segmentation of the Human Placenta from Motion Corrupted MRI,” in MICCAI , 2016
2016
Later among the works it cites.
K. Kamnitsas, C. Ledig, V. F. J. Newcombe, J. P. Simpson, A. D. Kane, D. K. Menon, D. Rueckert, and B. Glocker, “Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation,” Medical Image Analysis , vol. 36, pp. 61–78, 2017
2017
Closest in time.
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs,” PAMI , vol. PP, no. 99, pp. 1–1, 2017
2017
Closest in time.
I. Luengo, M. C. Darrow, M. C. Spink, Y. Sun, W. Dai, C. Y. He, W. Chiu, T. Pridmore, A. W. Ashton, E. M. Duke, M. Basham, and A. P. French, “SuRVoS: Super-Region Volume Segmentation workbench,” Journal of Structural Biology , vol. 198, no. 1, pp. 43–53, 2017
2017
Closest in time.
W. Li, G. Wang, L. Fidon, S. Ourselin, M. J. Cardoso, and T. Vercauteren, “On the Compactness , Efficiency , and Representation of 3D Convolutional Networks : Brain Parcellation as a Pretext Task,” in IPMI , 2017
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…