Masked Autoencoders as Image Processors

Understand

Transformers have shown significant effectiveness for various vision tasks including both high-level vision and low-level vision.

• Recently, masked autoencoders (MAE) for feature pre-training have further unleashed the potential of Transformers, leading to state-of-the-art performances on various high-level vision tasks.
• However, the significance of MAE pre-training on low-level vision tasks has not been sufficiently explored.
• In this paper, we show that masked autoencoders are also scalable self-supervised learners for image processing tasks.