2019

Style Generator Inversion for Image Enhancement and Animation

Gabbay, Aviv, Hoshen, Yedid

Understand

One of the main motivations for training high quality image generative models is their potential use as tools for image manipulation.

Recently, generative adversarial networks (GANs) have been able to generate images of remarkable quality.
Unfortunately, adversarially-trained unconditional generator networks have not been successful as image priors.
One of the main requirements for a network to act as a generative image prior, is being able to generate every possible image from the target distribution.

Built on

A morphable model for the synthesis of 3d faces
V. Blanz, T. Vetter, et al · 1999
Earlier work this paper cites.
Automatic 3d face reconstruction from single images or video
P. Breuer, K.-I. Kim, W. Kienzle, B. Scholkopf, and V. Blanz · 2008
Earlier work this paper cites.
From learning models of natural image patches to whole image restoration
D. Zoran and Y. Weiss · 2011
Earlier work this paper cites.
Nice: Non-linear independent components estimation
Original
L. Dinh, D. Krueger, and Y. Bengio · 2014
Earlier work this paper cites.
Generative adversarial nets
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio · 2014
Earlier work this paper cites.
Auto-encoding variational bayes
D. P. Kingma and M. Welling · 2014
Earlier work this paper cites.
From facial parts responses to face detection: A deep learning approach
S. Yang, P. Luo, C. C. Loy, and X. Tang · 2015
Earlier work this paper cites.
InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets
X. Chen, X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel · 2016
Earlier work this paper cites.
Unsupervised representation learning with deep convolutional generative adversarial networks
A. Radford, L. Metz, and S. Chintala · 2016
Earlier work this paper cites.
Generative image modeling using style and structure adversarial networks
X. Wang and A. Gupta · 2016
Earlier work this paper cites.
Generative visual manipulation on the natural image manifold
J.-Y. Zhu, P. Krähenbühl, E. Shechtman, and A. A. Efros · 2016
Earlier work this paper cites.
Wasserstein gan
M. Arjovsky, S. Chintala, and L. Bottou · 2017
Earlier work this paper cites.

Similar

Bringing portraits to life
H. Averbuch-Elor, D. Cohen-Or, J. Kopf, and M. F. Cohen · 2017
Cited alongside, same era.
Improved training of wasserstein gans
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville · 2017
Cited alongside, same era.
Arbitrary style transfer in real-time with adaptive instance normalization
X. Huang and S. Belongie · 2017
Cited alongside, same era.
Image-to-image translation with conditional adversarial networks
P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros · 2017
Cited alongside, same era.
Progressive growing of gans for improved quality, stability, and variation
Original
T. Karras, T. Aila, S. Laine, and J. Lehtinen · 2017
Cited alongside, same era.
Optimizing the latent space of generative networks
P. Bojanowski, A. Joulin, D. Lopez-Paz, and A. Szlam · 2018
Cited alongside, same era.
Large scale gan training for high fidelity natural image synthesis
Original
A. Brock, J. Donahue, and K. Simonyan · 2018
Cited alongside, same era.
Non-adversarial mapping with vaes
Y. Hoshen · 2018
Cited alongside, same era.
Non-adversarial image synthesis with generative latent nearest neighbors
Original
Y. Hoshen and J. Malik · 2018
Cited alongside, same era.
Nam: Non-adversarial unsupervised domain mapping
Y. Hoshen and L. Wolf · 2018
Cited alongside, same era.
Multimodal unsupervised image-to-image translation
X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz · 2018
Cited alongside, same era.

Then

A style-based generator architecture for generative adversarial networks
Original
T. Karras, S. Laine, and T. Aila · 2018
Later among the works it cites.
Glow: Generative flow with invertible 1x1 convolutions
Original
D. P. Kingma and P. Dhariwal · 2018
Later among the works it cites.
Which training methods for gans do actually converge?
L. Mescheder, S. Nowozin, and A. Geiger · 2018
Later among the works it cites.
Spectral normalization for generative adversarial networks
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida · 2018
Later among the works it cites.
Structured gans
I. Peleg and L. Wolf · 2018
Later among the works it cites.
Ganimation: Anatomically-aware facial animation from a single image
A. Pumarola, A. Agudo, A. M. Martinez, A. Sanfeliu, and F. Moreno-Noguer · 2018
Later among the works it cites.
Wasserstein auto-encoders
I. Tolstikhin, O. Bousquet, S. Gelly, and B. Schoelkopf · 2018
Later among the works it cites.
Deep image prior
D. Ulyanov, A. Vedaldi, and V. Lempitsky · 2018
Later among the works it cites.
Video-to-video synthesis
Original
T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, G. Liu, A. Tao, J. Kautz, and B. Catanzaro · 2018
Later among the works it cites.
Self-attention generative adversarial networks
Original
H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena · 2018
Later among the works it cites.
The unreasonable effectiveness of deep features as a perceptual metric
Original
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…