Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation

Understand

In this paper, to remedy this deficiency, we propose a Linear Attention Mechanism which is approximate to dot-product attention with much less memory and computational costs.

• The efficient design makes the incorporation between attention mechanisms and neural networks more flexible and versatile.
• Experiments conducted on semantic segmentation demonstrated the effectiveness of linear attention mechanism.
• Code is available at https://github.com/lironui/Linear-Attention-Mechanism.