SwinGNN: Rethinking Permutation Invariance in Diffusion Models for Graph Generation

Understand

Diffusion models based on permutation-equivariant networks can learn permutation-invariant distributions for graph data.

• However, in comparison to their non-invariant counterparts, we have found that these invariant models encounter greater learning challenges since 1) their effective target distributions exhibit more modes; 2) their optimal one-step denoising scores are the score functions of Gaussian mixtures with more components.
• Motivated by this analysis, we propose a non-invariant diffusion model, called $\textit{SwinGNN}$, which employs an efficient edge-to-edge 2-WL message passing network and utilizes shifted window based self-attention inspired by SwinTransformers.
• Further, through systematic ablations, we identify several critical training and sampling techniques that significantly improve the sample quality of graph generation.