2019

Revisiting Precision and Recall Definition for Generative Model Evaluation

Simon, Loïc, Webster, Ryan, Rabin, Julien

Understand

In this article we revisit the definition of Precision-Recall (PR) curves for generative models proposed by Sajjadi et al.

(arXiv:1806.00035).
Rather than providing a scalar for generative quality, PR curves distinguish mode-collapse (poor recall) and bad quality (poor precision).
We first generalize their formulation to arbitrary measures, hence removing any restriction to finite support.

Generative adversarial nets
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y · 2014
Earlier work this paper cites.
Auto-encoding variational bayes
Kingma, D. P. and Welling, M · 2014
Earlier work this paper cites.
Deep face recognition
Parkhi, O. M., Vedaldi, A., and Zisserman, A · 2015
Earlier work this paper cites.
Unsupervised representation learning with deep convolutional generative adversarial networks
Original
Radford, A., Metz, L., and Chintala, S · 2015
Earlier work this paper cites.
Linking losses for density ratio and class-probability estimation
Menon, A. and Ong, C. S · 2016
Earlier work this paper cites.
Generative adversarial text to image synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., and Lee, H · 2016
Earlier work this paper cites.
Improved techniques for training gans
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., and Chen, X · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z · 2016
Earlier work this paper cites.

Gans trained by a two time-scale update rule converge to a local nash equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S · 2017
Cited alongside, same era.
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., and Ishikawa, H · 2017
Cited alongside, same era.
Revisiting classifier two-sample tests
Lopez-Paz, D. and Oquab, M · 2017
Cited alongside, same era.
Unrolled generative adversarial networks
Metz, L., Poole, B., Pfau, D., and Sohl-Dickstein, J · 2017
Cited alongside, same era.
Segan: Speech enhancement generative adversarial network
Pascual, S., Bonafonte, A., and Serra, J · 2017
Cited alongside, same era.
A norm for signed measures. application to non local transport equation with source term
Piccoli, B., Rossi, F., and Tournus, M · 2017
Cited alongside, same era.
Quantitatively evaluating gans with divergences proposed for training
Original
Im, D. J., Ma, H., Taylor, G., and Branson, K · 2018
Cited alongside, same era.
Progressive growing of GANs for improved quality, stability, and variation
Karras, T., Aila, T., Laine, S., and Lehtinen, J · 2018
Cited alongside, same era.
Wasserstein generative adversarial networks
Arjovsky, M., Chintala, S., and Bottou, L
Cited in the paper.
Wasserstein gan
Original
Arjovsky, M., Chintala, S., and Bottou, L
Cited in the paper.

Word translation without parallel data
Lample, G., Conneau, A., Ranzato, M., Denoyer, L., and Jégou, H · 2018
Later among the works it cites.
Pacgan: The power of two samples in generative adversarial networks
Lin, Z., Khetan, A., Fanti, G., and Oh, S · 2018
Later among the works it cites.
Are gans created equal? a large-scale study
Lucic, M., Kurach, K., Michalski, M., Gelly, S., and Bousquet, O · 2018
Later among the works it cites.
Which training methods for gans do actually converge?
Mescheder, L., Geiger, A., and Nowozin, S · 2018
Later among the works it cites.
Assessing generative models via precision and recall
Sajjadi, M. S. M., Bachem, O., Lucic, M., Bousquet, O., and Gelly, S · 2018
Later among the works it cites.
Large scale GAN training for high fidelity natural image synthesis
Brock, A., Donahue, J., and Simonyan, K · 2019
Closest in time.
Detecting overfitting of deep generative networks via latent recovery
Webster, R., Rabin, J., Simon, L., and Jurie, F · 2019
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…