2018

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Wang, Xintao, Yu, Ke, Wu, Shixiang et al.

Understand

The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution.

However, the hallucinated details are often accompanied with unpleasant artifacts.
To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN).
In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit.

Built on

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics
Martin, D., Fowlkes, C., Tal, D., Malik, J.: · 2001
Earlier work this paper cites.
On single image scale-up using sparse-representations
Zeyde, R., Elad, M., Protter, M.: · 2010
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Glorot, X., Bengio, Y.: · 2010
Earlier work this paper cites.
Low-complexity single-image super-resolution based on nonnegative neighbor embedding
Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: · 2012
Earlier work this paper cites.
Making a completely blind image quality analyzer
Mittal, A., Soundararajan, R., Bovik, A.C.: · 2013
Earlier work this paper cites.
Learning a deep convolutional network for image super-resolution
Dong, C., Loy, C.C., He, K., Tang, X.: · 2014
Earlier work this paper cites.
Generative adversarial nets
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Original
Simonyan, K., Zisserman, A.: · 2014
Earlier work this paper cites.
Super-resolution with deep convolutional sufficient statistics
Bruna, J., Sprechmann, P., LeCun, Y.: · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Ioffe, S., Szegedy, C.: · 2015
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., Sun, J.: · 2015
Earlier work this paper cites.
Texture synthesis using convolutional neural networks
Gatys, L., Ecker, A.S., Bethge, M.: · 2015
Earlier work this paper cites.
Material recognition in the wild with the materials in context database
Bell, S., Upchurch, P., Snavely, N., Bala, K.: · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D., Ba, J.: · 2015
Earlier work this paper cites.
Single image super-resolution from transformed self-exemplars
Huang, J.B., Singh, A., Ahuja, N.: · 2015
Earlier work this paper cites.
Accurate image super-resolution using very deep convolutional networks
Kim, J., Kwon Lee, J., Mu Lee, K.: · 2016
Earlier work this paper cites.

Similar

Deeply-recursive convolutional network for image super-resolution
Kim, J., Kwon Lee, J., Mu Lee, K.: · 2016
Cited alongside, same era.
Perceptual losses for real-time style transfer and super-resolution
Johnson, J., Alahi, A., Fei-Fei, L.: · 2016
Cited alongside, same era.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., Sun, J.: · 2016
Cited alongside, same era.
Inception-v4, inception-resnet and the impact of residual connections on learning
Original
Szegedy, C., Ioffe, S., Vanhoucke, V.: · 2016
Cited alongside, same era.
Image super-resolution using deep convolutional networks
Dong, C., Loy, C.C., He, K., Tang, X.: · 2016
Cited alongside, same era.
Seven ways to improve example-based single image super resolution
Timofte, R., Rothe, R., Van Gool, L.: · 2016
Cited alongside, same era.
Photo-realistic single image super-resolution using a generative adversarial network
Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.: · 2017
Cited alongside, same era.
Deep laplacian pyramid networks for fast and accurate super-resolution
Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: · 2017
Cited alongside, same era.
Image super-resolution via deep recursive residual network
Tai, Y., Yang, J., Liu, X.: · 2017
Cited alongside, same era.
Memnet: A persistent memory network for image restoration
Tai, Y., Yang, J., Liu, X., Xu, C.: · 2017
Cited alongside, same era.
Enhancenet: Single image super-resolution through automated texture synthesis
Sajjadi, M.S., Schölkopf, B., Hirsch, M.: · 2017
Cited alongside, same era.
Enhanced deep residual networks for single image super-resolution
Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: · 2017
Cited alongside, same era.

Then

Densely connected convolutional networks
Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: · 2017
Later among the works it cites.
Deep multi-scale convolutional neural network for dynamic scene deblurring
Nah, S., Kim, T.H., Lee, K.M.: · 2017
Later among the works it cites.
Residual networks of residual networks: Multilevel residual networks
Zhang, K., Sun, M., Han, X., Yuan, X., Guo, L., Liu, T.: · 2017
Later among the works it cites.
Ntire 2017 challenge on single image super-resolution: Dataset and study
Agustsson, E., Timofte, R.: · 2017
Later among the works it cites.
Ntire 2017 challenge on single image super-resolution: Methods and results
Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., Zhang, L., Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M., et al.: · 2017
Later among the works it cites.
The relativistic discriminator: a key element missing from standard gan
Original
Jolicoeur-Martineau, A.: · 2018
Closest in time.
The pirm challenge on perceptual super resolution
Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., Zelnik-Manor, L.: · 2018
Closest in time.
Deep backprojection networks for super-resolution
Haris, M., Shakhnarovich, G., Ukita, N.: · 2018
Closest in time.
Residual dense network for image super-resolution
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: · 2018
Closest in time.
Image super-resolution using very deep residual channel attention networks
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: · 2018
Closest in time.
Recovering realistic texture in image super-resolution by deep spatial feature transform
Wang, X., Yu, K., Dong, C., Loy, C.C.: · 2018
Closest in time.
Crafting a toolchain for image restoration by deep reinforcement learning
Yu, K., Dong, C., Lin, L., Loy, C.C.: · 2018
Closest in time.
Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks
Yuan, Y., Liu, S., Zhang, J., Zhang, Y., Dong, C., Lin, L.: · 2018
Closest in time.
Maintaining natural image statistics with the contextual loss
Original
Mechrez, R., Talmi, I., Shama, F., Zelnik-Manor, L.: · 2018
Closest in time.
Spectral normalization for generative adversarial networks
Original
Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…