Parsing as Pretraining

Understand

Recent analyses suggest that encoders pretrained for language modeling capture certain morpho-syntactic structure.

• However, probing frameworks for word vectors still do not report results on standard setups such as constituent and dependency parsing.
• This paper addresses this problem and does full parsing (on English) relying only on pretraining architectures -- and no decoding.
• We first cast constituent and dependency parsing as sequence tagging.