gSwin: Gated MLP Vision Model with Hierarchical Structure of Shifted Window

Understand

Following the success in language domain, the self-attention mechanism (transformer) is adopted in the vision domain and achieving great success recently.

• Additionally, as another stream, multi-layer perceptron (MLP) is also explored in the vision domain.
• These architectures, other than traditional CNNs, have been attracting attention recently, and many methods have been proposed.
• As one that combines parameter efficiency and performance with locality and hierarchy in image recognition, we propose gSwin, which merges the two streams; Swin Transformer and (multi-head) gMLP.