ViTAR: Vision Transformer with Any Resolution

Understand

This paper tackles a significant challenge faced by Vision Transformers (ViTs): their constrained scalability across different image resolutions.

• Typically, ViTs experience a performance decline when processing resolutions different from those seen during training.
• Our work introduces two key innovations to address this issue.
• Firstly, we propose a novel module for dynamic resolution adjustment, designed with a single Transformer block, specifically to achieve highly efficient incremental token integration.