High-Fidelity Pluralistic Image Completion with Transformers

Understand

Image completion has made tremendous progress with convolutional neural networks (CNNs), because of their powerful texture modeling capacity.

• However, due to some inherent properties (e.g., local inductive prior, spatial-invariant kernels), CNNs do not perform well in understanding global structures or naturally support pluralistic completion.
• Recently, transformers demonstrate their power in modeling the long-term relationship and generating diverse results, but their computation complexity is quadratic to input length, thus hampering the application in processing high-resolution images.
• This paper brings the best of both worlds to pluralistic image completion: appearance prior reconstruction with transformer and texture replenishment with CNN.