A Survey on Visual Transformer

Understand

Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism.

• Thanks to its strong representation capabilities, researchers are looking at ways to apply transformer to computer vision tasks.
• In a variety of visual benchmarks, transformer-based models perform similar to or better than other types of networks such as convolutional and recurrent neural networks.
• Given its high performance and less need for vision-specific inductive bias, transformer is receiving more and more attention from the computer vision community.