Learning Unitary Operators with Help From u(n)

Understand

A major challenge in the training of recurrent neural networks is the so-called vanishing or exploding gradient problem.

• The use of a norm-preserving transition operator can address this issue, but parametrization is challenging.
• In this work we focus on unitary operators and describe a parametrization using the Lie algebra $\mathfrak{u}(n)$ associated with the Lie group $U(n)$ of $n \times n$ unitary matrices.
• The exponential map provides a correspondence between these spaces, and allows us to define a unitary matrix using $n^2$ real coefficients relative to a basis of the Lie algebra.