Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators

Understand

Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning.

• However, their desirable characteristics such as analytic invertibility come at the cost of restricting the functional forms.
• This poses a question on their representation power: are CF-INNs universal approximators for invertible functions? Without a universality, there could be a well-behaved invertible transformation that the CF-INN can never approximate, hence it would render the model class unreliable.
• We answer this question by showing a convenient criterion: a CF-INN is universal if its layers contain affine coupling and invertible linear functions as special cases.