Learning Input and Recurrent Weight Matrices in Echo State Networks

Understand

Echo State Networks (ESNs) are a special type of the temporally deep network model, the Recurrent Neural Network (RNN), where the recurrent matrix is carefully designed and both the recurrent and input matrices are fixed.

• An ESN uses the linearity of the activation function of the output units to simplify the learning of the output matrix.
• In this paper, we devise a special technique that take advantage of this linearity in the output units of an ESN, to learn the input and recurrent matrices.
• This has not been done in earlier ESNs due to their well known difficulty in learning those matrices.