Universal approximation property of invertible neural networks

Understand

Invertible neural networks (INNs) are neural network architectures with invertibility by design.

• Thanks to their invertibility and the tractability of Jacobian, INNs have various machine learning applications such as probabilistic modeling, generative modeling, and representation learning.
• However, their attractive properties often come at the cost of restricting the layer designs, which poses a question on their representation power: can we use these models to approximate sufficiently diverse functions? To answer this question, we have developed a general theoretical framework to investigate the representation power of INNs, building on a structure theorem of differential geometry.
• The framework simplifies the approximation problem of diffeomorphisms, which enables us to show the universal approximation properties of INNs.