LaMamba-Diff: Linear-Time High-Fidelity Diffusion Models Based on Local Attention and Mamba

Understand

Recent Transformer-based diffusion models have shown remarkable performance, largely attributed to the ability of the self-attention mechanism to accurately capture both global and local contexts by computing all-pair interactions among input tokens.

• However, their quadratic complexity poses significant computational challenges for long-sequence inputs.
• Conversely, a recent state space model called Mamba offers linear complexity by compressing a filtered global context into a hidden state.
• Despite its efficiency, compression inevitably leads to information loss of fine-grained local dependencies among tokens, which are crucial for effective visual generative modeling.