MoCoViT: Mobile Convolutional Vision Transformer

Understand

Recently, Transformer networks have achieved impressive results on a variety of vision tasks.

• However, most of them are computationally expensive and not suitable for real-world mobile applications.
• In this work, we present Mobile Convolutional Vision Transformer (MoCoViT), which improves in performance and efficiency by introducing transformer into mobile convolutional networks to leverage the benefits of both architectures.
• Different from recent works on vision transformer, the mobile transformer block in MoCoViT is carefully designed for mobile devices and is very lightweight, accomplished through two primary modifications: the Mobile Self-Attention (MoSA) module and the Mobile Feed Forward Network (MoFFN).