Memorization Precedes Generation: Learning Unsupervised GANs with Memory Networks

Understand

We propose an approach to address two issues that commonly occur during training of unsupervised GANs.

• First, since GANs use only a continuous latent distribution to embed multiple classes or clusters of data, they often do not correctly handle the structural discontinuity between disparate classes in a latent space.
• Second, discriminators of GANs easily forget about past generated samples by generators, incurring instability during adversarial training.
• We argue that these two infamous problems of unsupervised GAN training can be largely alleviated by a learnable memory network to which both generators and discriminators can access.