Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings

Understand

Contextualized word embeddings (CWE) such as provided by ELMo (Peters et al., 2018), Flair NLP (Akbik et al., 2018), or BERT (Devlin et al., 2019) are a major recent innovation in NLP.

• CWEs provide semantic vector representations of words depending on their respective context.
• Their advantage over static word embeddings has been shown for a number of tasks, such as text classification, sequence tagging, or machine translation.
• Since vectors of the same word type can vary depending on the respective context, they implicitly provide a model for word sense disambiguation (WSD).