Constituency Parsing with a Self-Attentive Encoder

Understand

We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser.

• The use of attention makes explicit the manner in which information is propagated between different locations in the sentence, which we use to both analyze our model and propose potential improvements.
• For example, we find that separating positional and content information in the encoder can lead to improved parsing accuracy.
• Additionally, we evaluate different approaches for lexical representation.