MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer

Understand

Light-weight convolutional neural networks (CNNs) are the de-facto for mobile vision tasks.

• Their spatial inductive biases allow them to learn representations with fewer parameters across different vision tasks.
• However, these networks are spatially local.
• To learn global representations, self-attention-based vision trans-formers (ViTs) have been adopted.