Relaxing Bijectivity Constraints with Continuously Indexed Normalising Flows

Understand

We show that normalising flows become pathological when used to model targets whose supports have complicated topologies.

• In this scenario, we prove that a flow must become arbitrarily numerically noninvertible in order to approximate the target closely.
• This result has implications for all flow-based models, and especially Residual Flows (ResFlows), which explicitly control the Lipschitz constant of the bijection used.
• To address this, we propose Continuously Indexed Flows (CIFs), which replace the single bijection used by normalising flows with a continuously indexed family of bijections, and which can intuitively "clean up" mass that would otherwise be misplaced by a single bijection.