Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling

Understand

Diffusion models excel at generating photo-realistic images but come with significant computational costs in both training and sampling.

• While various techniques address these computational challenges, a less-explored issue is designing an efficient and adaptable network backbone for iterative refinement.
• Current options like U-Net and Vision Transformer often rely on resource-intensive deep networks and lack the flexibility needed for generating images at variable resolutions or with a smaller network than used in training.
• This study introduces LEGO bricks, which seamlessly integrate Local-feature Enrichment and Global-content Orchestration.