Visual Attention Methods in Deep Learning: An In-Depth Survey

Understand

Inspired by the human cognitive system, attention is a mechanism that imitates the human cognitive awareness about specific information, amplifying critical details to focus more on the essential aspects of data.

• Deep learning has employed attention to boost performance for many applications.
• Interestingly, the same attention design can suit processing different data modalities and can easily be incorporated into large networks.
• Furthermore, multiple complementary attention mechanisms can be incorporated into one network.