Analogies Explained: Towards Understanding Word Embeddings

A synopsis of linguistic theory, 1930-1955

Firth, J. R · 1957

Earlier work this paper cites.

Weighted low-rank approximations

Srebro, N. and Jaakkola, T · 2003

Earlier work this paper cites.

Word representations: a simple and general method for semi-supervised learning

Turian, J., Ratinov, L., and Bengio, Y · 2010

Earlier work this paper cites.

From frequency to meaning: Vector space models of semantics

Turney, P. D. and Pantel, P · 2010

Earlier work this paper cites.

Linguistic regularities in continuous space word representations

Mikolov, T., Yih, W.-t., and Zweig, G · 2013

Earlier work this paper cites.

Parsing with compositional vector grammars

Socher, R., Bauer, J., Manning, C. D., et al · 2013

Cited alongside, same era.

Glove: Global vectors for word representation

Pennington, J., Socher, R., and Manning, C · 2014

Cited alongside, same era.

A latent variable model approach to pmi-based word embeddings

Arora, S., Li, Y., Liang, Y., Ma, T., and Risteski, A · 2016

Cited alongside, same era.

Man is to computer programmer as woman is to homemaker? debiasing word embeddings

Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V., and Kalai, A. T · 2016

Cited alongside, same era.

Word embeddings as metric recovery in semantic spaces

Hashimoto, T. B., Alvarez-Melis, D., and Jaakkola, T. S · 2016

Cited alongside, same era.

Linguistic regularities in sparse and explicit word representations

Levy, O. and Goldberg, Y

Cited in the paper.

Neural word embedding as implicit matrix factorization

Levy, O. and Goldberg, Y

Cited in the paper.

Distributed representations of words and phrases and their compositionality

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., and Dean, J

Cited in the paper.

Issues in evaluating semantic spaces using word analogies

Original

Linzen, T · 2016

Later among the works it cites.

Skip-Gram - Zipf + Uniform = Vector Additivity

Gittens, A., Achlioptas, D., and Mahoney, M. W · 2017

Later among the works it cites.

Unshifting the PMI matrix

Le, M · 2017

Later among the works it cites.

Towards understanding linear word analogies

Original

Ethayarajh, K., Duvenaud, D., and Hirst, G · 2018

Later among the works it cites.

text8 wikipedia dump

Mahoney, M · 2019

Closest in time.