2020

Feature Quantization Improves GAN Training

Zhao, Yang, Li, Chunyuan, Yu, Ping et al.

Understand

The instability in GAN training has been a long-standing problem despite remarkable research efforts.

We identify that instability issues stem from difficulties of performing feature matching with mini-batch statistics, due to a fragile balance between the fixed target distribution and the progressively generated distribution.
In this work, we propose Feature Quantization (FQ) for the discriminator, to embed both true and fake data samples into a shared discrete space.
The quantized values of FQ are constructed as an evolving dictionary, which is consistent with feature statistics of the recent distribution history.

Built on

Consistency regularization for generative adversarial networks
Original
Zhang, H., Zhang, Z., Odena, A., and Lee, H · 1910
Earlier work this paper cites.
Analyzing and improving the image quality of styleGAN
Original
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., and Aila, T · 1912
Earlier work this paper cites.
Vector quantization
Gray, R · 1984
Earlier work this paper cites.
A new vector quantization clustering algorithm
Equitz, W. H · 1989
Earlier work this paper cites.
Integral probability metrics and their generating classes of functions
Müller, A · 1997
Earlier work this paper cites.
Visualizing data using t-SNE
Maaten, L. v. d. and Hinton, G · 2008
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
Distance learning in discriminative vector quantization
Schneider, P., Biehl, M., and Hammer, B · 2009
Earlier work this paper cites.
Generative adversarial nets
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y · 2014
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al · 2015
Earlier work this paper cites.
f-GAN: Training generative neural samplers using variational divergence minimization
Nowozin, S., Cseke, B., and Tomioka, R · 2016
Earlier work this paper cites.
Improved techniques for training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X · 2016
Earlier work this paper cites.
Amortised MAP inference for image super-resolution
Original
Sønderby, C. K., Caballero, J., Theis, L., Shi, W., and Huszár, F · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z · 2016
Earlier work this paper cites.
Learning deep features for discriminative localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A · 2016
Earlier work this paper cites.

Similar

Wasserstein GAN
Original
Arjovsky, M., Chintala, S., and Bottou, L · 2017
Cited alongside, same era.
Many paths to equilibrium: Gans do not need to decrease a divergence at every step
Original
Fedus, W., Rosca, M., Lakshminarayanan, B., Dai, A. M., Mohamed, S., and Goodfellow, I · 2017
Cited alongside, same era.
Improved training of wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. C · 2017
Cited alongside, same era.
GANs trained by a two time-scale update rule converge to a local nash equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S · 2017
Cited alongside, same era.
Image-to-image translation with conditional adversarial networks
Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A. A · 2017
Cited alongside, same era.
On convergence and stability of GANs
Original
Kodali, N., Abernethy, J., Hays, J., and Kira, Z · 2017
Cited alongside, same era.
Unsupervised image-to-image translation networks
Liu, M.-Y., Breuel, T., and Kautz, J · 2017
Cited alongside, same era.
Least squares generative adversarial networks
Mao, X., Li, Q., Xie, H., Lau, R. Y., Wang, Z., and Paul Smolley, S · 2017
Cited alongside, same era.
McGAN: Mean and covariance feature matching gan
Original
Mroueh, Y., Sercu, T., and Goel, V · 2017
Cited alongside, same era.
Neural discrete representation learning
Oord, A. v. d., Vinyals, O., and Kavukcuoglu, K · 2017
Cited alongside, same era.
Adversarial discriminative domain adaptation
Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T · 2017
Cited alongside, same era.
StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks
Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., and Metaxas, D. N · 2017
Cited alongside, same era.

Then

Progressive growing of GANs for improved quality, stability, and variation
Karras, T., Aila, T., Laine, S., and Lehtinen, J · 2018
Later among the works it cites.
Memorization precedes generation: Learning unsupervised GANs with memory networks
Original
Kim, Y., Kim, M., and Kim, G · 2018
Later among the works it cites.
Diverse image-to-image translation via disentangled representations
Lee, H.-Y., Tseng, H.-Y., Huang, J.-B., Singh, M., and Yang, M.-H · 2018
Later among the works it cites.
Generative adversarial network training is a continual learning problem
Original
Liang, K. J., Li, C., Wang, G., and Carin, L · 2018
Later among the works it cites.
Which training methods for GANs do actually converge?
Original
Mescheder, L., Geiger, A., and Nowozin, S · 2018
Later among the works it cites.
Spectral normalization for generative adversarial networks
Original
Miyato, T., Kataoka, T., Koyama, M., and Yoshida, Y · 2018
Later among the works it cites.
Variational discriminator bottleneck: Improving imitation learning, inverse rl, and gans by constraining information flow
Original
Peng, X. B., Kanazawa, A., Toyer, S., Abbeel, P., and Levine, S · 2018
Later among the works it cites.
On self modulation for generative adversarial networks
Chen, T., Lucic, M., Houlsby, N., and Gelly, S · 2019
Later among the works it cites.
Twin auxilary classifiers GAN
Gong, M., Xu, Y., Li, C., Zhang, K., and Batmanghelich, K · 2019
Later among the works it cites.
Gpipe: Efficient training of giant neural networks using pipeline parallelism
Huang, Y., Cheng, Y., Bapna, A., Firat, O., Chen, D., Chen, M., Lee, H., Ngiam, J., Le, Q. V., Wu, Y., et al · 2019
Later among the works it cites.
Adversarial learning of a sampler based on an unnormalized distribution
Li, C., Bai, K., Li, J., Wang, G., Chen, C., and Carin, L · 2019
Later among the works it cites.
Improving MMD-GAN training with repulsive loss function
Wang, W., Sun, Y., and Halgamuge, S · 2019
Later among the works it cites.
DM-GAN: Dynamic memory generative adversarial networks for text-to-image synthesis
Zhu, M., Pan, P., Chen, W., and Yang, Y · 2019
Later among the works it cites.
Gan memory with no forgetting
Original
Cong, Y., Zhao, M., Li, J., Wang, S., and Carin, L · 2020
Closest in time.
U-gat-it: unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation
Kim, J., Kim, M., Kang, H., and Lee, K · 2020
Closest in time.
On leveraging pretrained gans for limited-data generation
Original
Zhao, M., Cong, Y., and Carin, L · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…