Multi-channel Encoder for Neural Machine Translation

Understand

Attention-based Encoder-Decoder has the effective architecture for neural machine translation (NMT), which typically relies on recurrent neural networks (RNN) to build the blocks that will be lately called by attentive reader during the decoding process.

• This design of encoder yields relatively uniform composition on source sentence, despite the gating mechanism employed in encoding RNN.
• On the other hand, we often hope the decoder to take pieces of source sentence at varying levels suiting its own linguistic structure: for example, we may want to take the entity name in its raw form while taking an idiom as a perfectly composed unit.
• Motivated by this demand, we propose Multi-channel Encoder (MCE), which enhances encoding components with different levels of composition.