Alternative structures for character-level RNNs

Understand

Recurrent neural networks are convenient and efficient models for language modeling.

• However, when applied on the level of characters instead of words, they suffer from several problems.
• In order to successfully model long-term dependencies, the hidden representation needs to be large.
• This in turn implies higher computational costs, which can become prohibitive in practice.