2017

Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks

Lai, Wei-Sheng, Huang, Jia-Bin, Ahuja, Narendra et al.

Understand

Convolutional neural networks have recently demonstrated high-quality reconstruction for single image super-resolution.

However, existing methods often require a large number of network parameters and entail heavy computational loads at runtime for generating high-accuracy super-resolution results.
In this paper, we propose the deep Laplacian Pyramid Super-Resolution Network for fast and accurate image super-resolution.
The proposed network progressively reconstructs the sub-band residuals of high-resolution images at multiple pyramid levels.

Built on

R. A. Bradley and M. E. Terry, “Rank analysis of incomplete block designs the method of paired comparisons,” Biometrika , vol. 39, no. 3-4, pp. 324–345, 1952
1952
Earlier work this paper cites.
P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Transactions on Communications , vol. 31, no. 4, pp. 532–540, 1983
1983
Earlier work this paper cites.
D. J. Heeger and J. R. Bergen, “Pyramid-based texture analysis/synthesis,” in 22nd Annual Conference on Computer Graphics and Interactive Techniques , 1995
1995
Earlier work this paper cites.
W. T. Freeman, T. R. Jones, and E. C. Pasztor, “Example-based super-resolution,” IEEE Computer Graphics and Applications , vol. 22, no. 2, pp. 56–65, 2002
2002
Earlier work this paper cites.
H. Chang, D.-Y. Yeung, and Y. Xiong, “Super-resolution through neighbor embedding,” in IEEE Conference on Computer Vision and Pattern Recognition , 2004
2004
Earlier work this paper cites.
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing , vol. 13, no. 4, pp. 600–612, 2004
2004
Earlier work this paper cites.
A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods,” International Journal of Computer Vision , vol. 61, no. 3, pp. 211–231, 2005
2005
Earlier work this paper cites.
H. R. Sheikh, A. C. Bovik, and G. De Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Transactions on Image Processing , vol. 14, no. 12, pp. 2117–2128, 2005
2005
Earlier work this paper cites.
J. Yang, J. Wright, T. Huang, and Y. Ma, “Image super-resolution as sparse representation of raw image patches,” in IEEE Conference on Computer Vision and Pattern Recognition , 2008
2008
Earlier work this paper cites.
D. Glasner, S. Bagon, and M. Irani, “Super-resolution from a single image,” in IEEE International Conference on Computer Vision , 2009
2009
Earlier work this paper cites.
J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image super-resolution via sparse representation,” IEEE Transactions on Image Processing , vol. 19, no. 11, pp. 2861–2873, 2010
2010
Earlier work this paper cites.
C.-Y. Yang, J.-B. Huang, and M.-H. Yang, “Exploiting self-similarities for single frame super-resolution,” in Asia Conference on Computer Vision , 2010
2010
Earlier work this paper cites.
K. I. Kim and Y. Kwon, “Single-image super-resolution using sparse regression and natural image prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 32, no. 6, pp. 1127–1133, 2010
2010
Earlier work this paper cites.
R. Zeyde, M. Elad, and M. Protter, “On single image scale-up using sparse-representations,” in International Conference on Curves and Surfaces , 2010
2010
Earlier work this paper cites.
M. Rubinstein, D. Gutierrez, O. Sorkine, and A. Shamir, “A comparative study of image retargeting,” ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia) , vol. 29, no. 6, pp. 160:1–160:10, 2010
2010
Earlier work this paper cites.
G. Freedman and R. Fattal, “Image and video upscaling from local self-examples,” ACM Transactions on Graphics (Proceedings of SIGGRAPH) , vol. 30, no. 2, p. 12, 2011
2011
Earlier work this paper cites.
S. Paris, S. W. Hasinoff, and J. Kautz, “Local laplacian filters: Edge-aware image processing with a laplacian pyramid.” ACM Transactions on Graphics (Proceedings of SIGGRAPH) , vol. 30, no. 4, p. 68, 2011
2011
Earlier work this paper cites.
P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 33, no. 5, pp. 898–916, 2011
2011
Earlier work this paper cites.
M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, “Low-complexity single-image super-resolution based on nonnegative neighbor embedding,” in British Machine Vision Conference , 2012
2012
Earlier work this paper cites.

Similar

C.-Y. Yang and M.-H. Yang, “Fast direct super-resolution by simple functions,” in IEEE International Conference on Computer Vision , 2013
2013
Cited alongside, same era.
R. Timofte, V. Smet, and L. Gool, “Anchored neighborhood regression for fast example-based super-resolution,” in IEEE International Conference on Computer Vision , 2013
2013
Cited alongside, same era.
A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities improve neural network acoustic models,” in International Conference on Machine Learning , 2013
2013
Cited alongside, same era.
R. Timofte, V. De Smet, and L. Van Gool, “A+: Adjusted anchored neighborhood regression for fast super-resolution,” in Asia Conference on Computer Vision , 2014
2014
Cited alongside, same era.
A. Singh and N. Ahuja, “Super-resolution using sub-band self-similarity,” in Asia Conference on Computer Vision , 2014
2014
Cited alongside, same era.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Neural Information Processing Systems , 2014
2014
Cited alongside, same era.
C.-Y. Yang, C. Ma, and M.-H. Yang, “Single-image super-resolution: A benchmark,” in European Conference on Computer Vision , 2014
2014
Cited alongside, same era.
S. Schulter, C. Leistner, and H. Bischof, “Fast and accurate image upscaling with super-resolution forests,” in IEEE Conference on Computer Vision and Pattern Recognition , 2015
2015
Cited alongside, same era.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition , 2015
2015
Cited alongside, same era.
P. Fischer, A. Dosovitskiy, E. Ilg, P. Häusser, C. Hazırbaş, V. Golkov, P. van der Smagt, D. Cremers, and T. Brox, “FlowNet: Learning optical flow with convolutional networks,” in IEEE International Conference on Computer Vision , 2015
2015
Cited alongside, same era.
C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 38, no. 2, pp. 295–307, 2015
2015
Cited alongside, same era.
Z. Wang, D. Liu, J. Yang, W. Han, and T. Huang, “Deep networks for image super-resolution with sparse prior,” in IEEE International Conference on Computer Vision , 2015
2015
Cited alongside, same era.

Then

J. Kim, J. K. Lee, and K. M. Lee, “Accurate image super-resolution using very deep convolutional networks,” in IEEE Conference on Computer Vision and Pattern Recognition , 2016
2016
Later among the works it cites.
J. Kim, J. K. Lee, and K. M. Lee, “Deeply-recursive convolutional network for image super-resolution,” in IEEE Conference on Computer Vision and Pattern Recognition , 2016
2016
Later among the works it cites.
W. Shi, J. Caballero, F. Huszar, J. Totz, A. Aitken, R. Bishop, D. Rueckert, and Z. Wang, “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network.” in IEEE Conference on Computer Vision and Pattern Recognition , 2016
2016
Later among the works it cites.
C. Dong, C. C. Loy, and X. Tang, “Accelerating the super-resolution convolutional neural network,” in European Conference on Computer Vision , 2016
2016
Later among the works it cites.
G. Ghiasi and C. C. Fowlkes, “Laplacian pyramid reconstruction and refinement for semantic segmentation,” in European Conference on Computer Vision , 2016
2016
Later among the works it cites.
D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context encoders: Feature learning by inpainting,” in IEEE Conference on Computer Vision and Pattern Recognition , 2016
2016
Later among the works it cites.
X. Yu and F. Porikli, “Ultra-resolving face images by discriminative generative networks,” in European Conference on Computer Vision , 2016
2016
Later among the works it cites.
J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in European Conference on Computer Vision , 2016
2016
Later among the works it cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in European Conference on Computer Vision , 2016
2016
Later among the works it cites.
Y. Matsui, K. Ito, Y. Aramaki, T. Yamasaki, and K. Aizawa, “Sketch-based manga retrieval using manga109 dataset,” Multimedia Tools and Applications , pp. 1–28, 2016
2016
Later among the works it cites.
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” 2016
2016
Later among the works it cites.
W.-S. Lai, J.-B. Huang, Z. Hu, N. Ahuja, and M.-H. Yang, “A comparative study for single image blind deblurring,” in IEEE Conference on Computer Vision and Pattern Recognition , 2016
2016
Later among the works it cites.
Y. Tai, J. Yang, and X. Liu, “Image super-resolution via deep recursive residual network,” in IEEE Conference on Computer Vision and Pattern Recognition , 2017
2017
Closest in time.
W.-S. Lai, J.-B. Huang, N. Ahuja, and M.-H. Yang, “Deep laplacian pyramid networks for fast and accurate super-resolution,” in IEEE Conference on Computer Vision and Pattern Recognition , 2017
2017
Closest in time.
B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, “Enhanced deep residual networks for single image super-resolution,” in IEEE Conference on Computer Vision and Pattern Recognition Workshops , 2017
2017
Closest in time.
R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang, B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, “NTIRE 2017 challenge on single image super-resolution: Methods and results,” in IEEE Conference on Computer Vision and Pattern Recognition Workshops , 2017
2017
Closest in time.
Y. Li, S. Liu, J. Yang, and M.-H. Yang, “Generative face completion,” in IEEE Conference on Computer Vision and Pattern Recognition , 2017
2017
Closest in time.
C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” in IEEE Conference on Computer Vision and Pattern Recognition , 2017
2017
Closest in time.
R. Timofte, E. Agustsson, L. V. Gool, M. Yang, L. Zhang, B. Lim, S. Son, H. Kim, S. Nah, K. M. Lee, X. Wang, Y. Tian, K. Yu, Y. Zhang, S. Wu, C. Dong, L. Lin, Y. Qiao, C. C. Loy, W. Bae, J. Yoo, Y. Han, J. C. Ye, J. Choi, M. Kim, Y. Fan, J. Yu, W. Han, D. Liu, H. Yu, Z. Wang, H. Shi, X. Wang, T. S. Huang, Y. Chen, K. Zhang, W. Zuo, Z. Tang, L. Luo, S. Li, M. Fu, L. Cao, W. Heng, G. Bui, T. Le, Y. Duan, D. Tao, R. Wang, X. Lin, J. Pang, J. Xu, Y. Zhao, X. Xu, J. Pan, D. Sun, Y. Zhang, X. Song, Y. Dai, X. Qin, X. Huynh, T. Guo, H. S. Mousavi, T. H. Vu, V. Monga, C. Cruz, K. Egiazarian, V. Katkovnik, R. Mehta, A. K. Jain, A. Agarwalla, C. V. S. Praveen, R. Zhou, H. Wen, C. Zhu, Z. Xia, Z. Wang, and Q. Guo, “Ntire 2017 challenge on single image super-resolution: Methods and results.”
Original
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…