UPANets: Learning from the Universal Pixel Attention Networks

Understand

Among image classification, skip and densely-connection-based networks have dominated most leaderboards.

• Recently, from the successful development of multi-head attention in natural language processing, it is sure that now is a time of either using a Transformer-like model or hybrid CNNs with attention.
• However, the former need a tremendous resource to train, and the latter is in the perfect balance in this direction.
• In this work, to make CNNs handle global and local information, we proposed UPANets, which equips channel-wise attention with a hybrid skip-densely-connection structure.