Linear Ensembles of Word Embedding Models

Understand

This paper explores linear methods for combining several word embedding models into an ensemble.

• We construct the combined models using an iterative method based on either ordinary least squares regression or the solution to the orthogonal Procrustes problem.
• We evaluate the proposed approaches on Estonian---a morphologically complex language, for which the available corpora for training word embeddings are relatively small.
• We compare both combined models with each other and with the input word embedding models using synonym and analogy tests.