Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention

Understand

Recently, Transformers have shown promising performance in various vision tasks.

• To reduce the quadratic computation complexity caused by the global self-attention, various methods constrain the range of attention within a local region to improve its efficiency.
• Consequently, their receptive fields in a single attention layer are not large enough, resulting in insufficient context modeling.
• To address this issue, we propose a Pale-Shaped self-Attention (PS-Attention), which performs self-attention within a pale-shaped region.