Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

Understand

We introduce a model for constructing vector representations of words by composing characters using bidirectional LSTMs.

• Relative to traditional word representation models that have independent vectors for each word type, our model requires only a single vector per character type and a fixed set of parameters for the compositional model.
• Despite the compactness of this model and, more importantly, the arbitrary nature of the form-function relationship in language, our "composed" word representations yield state-of-the-art results in language modeling and part-of-speech tagging.
• Benefits over traditional baselines are particularly pronounced in morphologically rich languages (e.g., Turkish).