TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up

Understand

The recent explosive interest on transformers has suggested their potential to become powerful "universal" models for computer vision tasks, such as classification, detection, and segmentation.

• While those attempts mainly study the discriminative models, we explore transformers on some more notoriously difficult vision tasks, e.g., generative adversarial networks (GANs).
• Our goal is to conduct the first pilot study in building a GAN completely free of convolutions, using only pure transformer-based architectures.
• Our vanilla GAN architecture, dubbed TransGAN, consists of a memory-friendly transformer-based generator that progressively increases feature resolution, and correspondingly a multi-scale discriminator to capture simultaneously semantic contexts and low-level textures.