Shortcut Sequence Tagging

Understand

Deep stacked RNNs are usually hard to train.

• Adding shortcut connections across different layers is a common way to ease the training of stacked networks.
• However, extra shortcuts make the recurrent step more complicated.
• To simply the stacked architecture, we propose a framework called shortcut block, which is a marriage of the gating mechanism and shortcuts, while discarding the self-connected part in LSTM cell.