2020

Few-shot Image Generation with Elastic Weight Consolidation

Li, Yijun, Zhang, Richard, Lu, Jingwan et al.

Understand

Few-shot image generation seeks to generate more data of a given domain, with only few available training examples.

As it is unreasonable to expect to fully infer the distribution from just a few observations (e.g., emojis), we seek to leverage a large, related source domain as pretraining (e.g., human faces).
Thus, we wish to preserve the diversity of the source domain, while adapting to the appearance of the target.
We adapt a pretrained model, without introducing any additional parameters, to the few examples of the target domain.

Built on

Image analogies
A. Hertzmann, C. E. Jacobs, N. Oliver, B. Curless, and D. H. Salesin · 2001
Earlier work this paper cites.
Generative adversarial nets
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio · 2014
Earlier work this paper cites.
Deep generative image models using a Laplacian pyramid of adversarial networks
E. L. Denton, S. Chintala, R. Fergus, et al · 2015
Earlier work this paper cites.
Human-level concept learning through probabilistic program induction
B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum · 2015
Earlier work this paper cites.
Deep learning face attributes in the wild
Z. Liu, P. Luo, X. Wang, and X. Tang · 2015
Earlier work this paper cites.
Unsupervised representation learning with deep convolutional generative adversarial networks
Original
A. Radford, L. Metz, and S. Chintala · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al · 2015
Earlier work this paper cites.
Fast adaptation in generative models with generative matching networks
Original
S. Bartunov and D. P. Vetrov · 2016
Earlier work this paper cites.
Image style transfer using convolutional neural networks
L. A. Gatys, A. S. Ecker, and M. Bethge · 2016
Earlier work this paper cites.
Perceptual losses for real-time style transfer and super-resolution
J. Johnson, A. Alahi, and L. Fei-Fei · 2016
Earlier work this paper cites.
One-shot generalization in deep generative models
Original
D. J. Rezende, S. Mohamed, I. Danihelka, K. Gregor, and D. Wierstra · 2016
Earlier work this paper cites.
Unsupervised cross-domain image generation
Original
Y. Taigman, A. Polyak, and L. Wolf · 2016
Earlier work this paper cites.
Matching networks for one shot learning
O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al · 2016
Earlier work this paper cites.
Model-agnostic meta-learning for fast adaptation of deep networks
C. Finn, P. Abbeel, and S. Levine · 2017
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter · 2017
Earlier work this paper cites.
Unsupervised cross-domain image generation
X. Hua, D. Rempe, and H. Zhang · 2017
Earlier work this paper cites.
Arbitrary style transfer in real-time with adaptive instance normalization
Original
X. Huang and S. Belongie · 2017
Earlier work this paper cites.

Similar

Image-to-image translation with conditional adversarial networks
P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros · 2017
Cited alongside, same era.
Progressive growing of gans for improved quality, stability, and variation
Original
T. Karras, T. Aila, S. Laine, and J. Lehtinen · 2017
Cited alongside, same era.
Overcoming catastrophic forgetting in neural networks
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al · 2017
Cited alongside, same era.
Universal style transfer via feature transforms
Y. Li, C. Fang, J. Yang, Z. Wang, X. Lu, and M.-H. Yang · 2017
Cited alongside, same era.
Learning without forgetting
Z. Li and D. Hoiem · 2017
Cited alongside, same era.
Gradient episodic memory for continual learning
D. Lopez-Paz and M. Ranzato · 2017
Cited alongside, same era.
A tutorial on fisher information
A. Ly, M. Marsman, J. Verhagen, R. P. Grasman, and E.-J. Wagenmakers · 2017
Cited alongside, same era.
Few-shot autoregressive density estimation: Towards learning to learn distributions
Original
S. Reed, Y. Chen, T. Paine, A. v. d. Oord, S. Eslami, D. Rezende, O. Vinyals, and N. de Freitas · 2017
Cited alongside, same era.
Continual learning in generative adversarial nets
Original
A. Seff, A. Beatson, D. Suo, and H. Liu · 2017
Cited alongside, same era.
Prototypical networks for few-shot learning
J. Snell, K. Swersky, and R. Zemel · 2017
Cited alongside, same era.
Continual learning through synaptic intelligence
F. Zenke, B. Poole, and S. Ganguli · 2017
Cited alongside, same era.
Unpaired image-to-image translation using cycle-consistent adversarial networks
J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros · 2017
Cited alongside, same era.

Then

Faceforensics: A large-scale video dataset for forgery detection in human faces
Original
A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner · 2018
Later among the works it cites.
Overcoming catastrophic forgetting with hard attention to the task
Original
J. Serrà, D. Suris, M. Miron, and A. Karatzoglou · 2018
Later among the works it cites.
Transferring gans: generating images from limited data
Y. Wang, C. Wu, L. Herranz, J. van de Weijer, A. Gonzalez-Garcia, and B. Raducanu · 2018
Later among the works it cites.
Memory replay gans: Learning to generate new categories without forgetting
C. Wu, L. Herranz, X. Liu, J. van de Weijer, B. Raducanu, et al · 2018
Later among the works it cites.
The unreasonable effectiveness of deep features as a perceptual metric
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang · 2018
Later among the works it cites.
A style-based generator architecture for generative adversarial networks
T. Karras, S. Laine, and T. Aila · 2019
Later among the works it cites.
Learning to self-train for semi-supervised few-shot classification
X. Li, Q. Sun, Y. Liu, Q. Zhou, S. Zheng, T.-S. Chua, and B. Schiele · 2019
Later among the works it cites.
Few-shot unsupervised image-to-image translation
M.-Y. Liu, X. Huang, A. Mallya, T. Karras, T. Aila, J. Lehtinen, and J. Kautz · 2019
Later among the works it cites.
Image generation from small datasets via batch statistics adaptation
A. Noguchi and T. Harada · 2019
Later among the works it cites.
Ancient painting to natural image: A new solution for painting processing
T. Qiao, W. Zhang, M. Zhang, Z. Ma, and D. Xu · 2019
Later among the works it cites.
The face of art: landmark detection and geometric style in portraits
J. Yaniv, Y. Newman, and A. Shamir · 2019
Later among the works it cites.
Few-shot adversarial learning of realistic neural talking head models
E. Zakharov, A. Shysheya, E. Burkov, and V. Lempitsky · 2019
Later among the works it cites.
Lifelong gan: Continual learning for conditional image generation
M. Zhai, L. Chen, F. Tung, J. He, M. Nawhal, and G. Mori · 2019
Later among the works it cites.
Stargan v2: Diverse image synthesis for multiple domains
Y. Choi, Y. Uh, J. Yoo, and J.-W. Ha · 2020
Closest in time.
Cnn-generated images are surprisingly easy to spot…for now
S.-Y. Wang, O. Wang, R. Zhang, A. Owens, and A. A. Efros · 2020
Closest in time.
Minegan: effective knowledge transfer from gans to target domains with few images
Y. Wang, A. Gonzalez-Garcia, D. Berga, L. Herranz, F. S. Khan, and J. van de Weijer · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…