2018

An Improved Evaluation Framework for Generative Adversarial Networks

Liu, Shaohui, Wei, Yi, Lu, Jiwen et al.

Understand

In this paper, we propose an improved quantitative evaluation framework for Generative Adversarial Networks (GANs) on generating domain-specific images, where we improve conventional evaluation methods on two levels: the feature representation and the evaluation metric.

Unlike most existing evaluation frameworks which transfer the representation of ImageNet inception model to map images onto the feature space, our framework uses a specialized encoder to acquire fine-grained domain-specific representation.
Moreover, for datasets with multiple classes, we propose Class-Aware Frechet Distance (CAFD), which employs a Gaussian mixture model on the feature space to better fit the multi-manifold feature distribution.
Experiments and analysis on both the feature level and the image level were conducted to demonstrate improvements of our proposed framework over the recently proposed state-of-the-art FID method.

Built on

Mardia’s test of multinormality
Mardia, K.V.: · 1985
Earlier work this paper cites.
K-sample anderson-darling tests
Scholz, F.W., Stephens, M.A.: · 1987
Earlier work this paper cites.
Visualizing data using t-sne
Maaten, L.v.d., Hinton, G.: · 2008
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G.: · 2009
Earlier work this paper cites.
Generative adversarial nets
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: · 2014
Earlier work this paper cites.
Mvn: an r package for assessing multivariate normality
Korkmaz, S., Goksuluk, D., Zararsiz, G.: · 2014
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Original
Goodfellow, I.J., Shlens, J., Szegedy, C.: · 2014
Earlier work this paper cites.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: · 2015
Earlier work this paper cites.
Training generative neural networks via maximum mean discrepancy optimization
Original
Dziugaite, G.K., Roy, D.M., Ghahramani, Z.: · 2015
Earlier work this paper cites.
Generative moment matching networks
Li, Y., Swersky, K., Zemel, R.: · 2015
Earlier work this paper cites.
Deep learning face attributes in the wild
Liu, Z., Luo, P., Wang, X., Tang, X.: · 2015
Earlier work this paper cites.
Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop
Original
Yu, F., Zhang, Y., Song, S., Seff, A., Xiao, J.: · 2015
Earlier work this paper cites.
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: · 2015
Earlier work this paper cites.
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., Chintala, S.: · 2016
Earlier work this paper cites.

Similar

A note on the evaluation of generative models
Theis, L., van den Oord, A., Bethge, M.: · 2016
Cited alongside, same era.
Improved techniques for training gans
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., Chen, X.: · 2016
Cited alongside, same era.
Generative visual manipulation on the natural image manifold
Zhu, J.Y., Krähenbühl, P., Shechtman, E., Efros, A.A.: · 2016
Cited alongside, same era.
Robust convolutional neural networks under adversarial noise
Jonghoon Jin, A.D., Culurciello, E.: · 2016
Cited alongside, same era.
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: · 2017
Cited alongside, same era.
Photo-realistic single image super-resolution using a generative adversarial network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W.: · 2017
Cited alongside, same era.
Wasserstein generative adversarial networks
Arjovsky, M., Chintala, S., Bottou, L.: · 2017
Cited alongside, same era.
BEGAN: boundary equilibrium generative adversarial networks
Original
Berthelot, D., Schumm, T., Metz, L.: · 2017
Cited alongside, same era.
Least squares generative adversarial networks
Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., Paul Smolley, S.: · 2017
Cited alongside, same era.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: · 2017
Cited alongside, same era.
Are GANs Created Equal? A Large-Scale Study
Lucic, M., Kurach, K., Michalski, M., Gelly, S., Bousquet, O.: · 2017
Cited alongside, same era.
Image-to-image translation with conditional adversarial networks
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: · 2017
Cited alongside, same era.

Then

Generalization and equilibrium in generative adversarial nets (gans)
Arora, S., Ge, R., Liang, Y., Ma, T., Zhang, Y.: · 2017
Later among the works it cites.
Mode regularized generative adversarial networks
Che, T., Li, Y., Jacob, A.P., Bengio, Y., Li, W.: · 2017
Later among the works it cites.
Energy-based generative adversarial network
Zhao, J., Mathieu, M., LeCun, Y.: · 2017
Later among the works it cites.
How to train your DRAGAN
Original
Kodali, N., Abernethy, J.D., Hays, J., Kira, Z.: · 2017
Later among the works it cites.
Improved training of wasserstein gans
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: · 2017
Later among the works it cites.
On the quantitative analysis of decoder-based generative models
Wu, Y., Burda, Y., Salakhutdinov, R., Grosse, R.: · 2017
Later among the works it cites.
Revisiting classifier two-sample tests
Lopez-Paz, D., Oquab, M.: · 2017
Later among the works it cites.
Variational Approaches for Auto-Encoding Generative Adversarial Networks
Rosca, M., Lakshminarayanan, B., Warde-Farley, D., Mohamed, S.: · 2017
Later among the works it cites.
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms (2017)
Original
Xiao, H., Rasul, K., Vollgraf, R.: · 2017
Later among the works it cites.
An empirical study on evaluation metrics of generative adversarial networks
Huang, G., Yuan, Y., Xu, Q., Guo, C., Sun, Y., Wu, F., Weinberger, K.: · 2018
Closest in time.
Quantitatively evaluating GANs with divergences proposed for training
Im, D.J., Ma, A.H., Taylor, G.W., Branson, K.: · 2018
Closest in time.
Pros and Cons of GAN Evaluation Measures
Borji, A.: · 2018
Closest in time.
MGAN: Training generative adversarial nets with multiple generators
Hoang, Q., Nguyen, T.D., Le, T., Phung, D.: · 2018
Closest in time.
A Note on the Inception Score
Barratt, S., Sharma, R.: · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…