Enhancing Machine Translation with Dependency-Aware Self-Attention

Understand

Most neural machine translation models only rely on pairs of parallel sentences, assuming syntactic information is automatically learned by an attention mechanism.

• In this work, we investigate different approaches to incorporate syntactic knowledge in the Transformer model and also propose a novel, parameter-free, dependency-aware self-attention mechanism that improves its translation quality, especially for long sentences and in low-resource scenarios.
• We show the efficacy of each approach on WMT English-German and English-Turkish, and WAT English-Japanese translation tasks.