Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss

Understand

Bidirectional long short-term memory (bi-LSTM) networks have recently proven successful for various NLP sequence modeling tasks, but little is known about their reliance to input representations, target languages, data set size, and label noise.

• We address these issues and evaluate bi-LSTMs with word, character, and unicode byte embeddings for POS tagging.
• We compare bi-LSTMs to traditional POS taggers across languages and data sizes.
• We also present a novel bi-LSTM model, which combines the POS tagging loss function with an auxiliary loss function that accounts for rare words.