Fetching the paper…

Reading the bibliography…

2017

word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA

Landgraf, Andrew J., Bellay, Jeremy

Original
PDF
Post on X

Understand

We show that the skip-gram formulation of word2vec trained with negative sampling is equivalent to a weighted logistic PCA.

This connection allows us to better understand the objective, compare it to other word embedding methods, and extend it to higher dimensional models.

word2vec Skip-Gram with Negative Sampling is a Weighted Logistic PCA · Around

Built on

Collaborative filtering for implicit feedback datasets
Hu, Y., Y. Koren, and C. Volinsky (2008) · 2008
Earlier work this paper cites.
Distributed representations of words and phrases and their compositionality
Mikolov, T., I. Sutskever, K. Chen, G. S. Corrado, and J. Dean (2013) · 2013
Earlier work this paper cites.
word2vec explained: Deriving Mikolov et al.’s negative-sampling word-embedding method
Original
Goldberg, Y. and O. Levy (2014) · 2014
Earlier work this paper cites.

Similar

Logistic matrix factorization for implicit feedback data
Johnson, C. C. (2014) · 2014
Cited alongside, same era.
Neural word embedding as implicit matrix factorization
Levy, O. and Y. Goldberg (2014) · 2014
Cited alongside, same era.
GloVe: Global vectors for word representation
Pennington, J., R. Socher, and C. D. Manning (2014) · 2014
Cited alongside, same era.

Then

Word embedding revisited: A new representation learning and explicit matrix factorization perspective
Li, Y., L. Xu, F. Tian, L. Jiang, X. Zhong, and E. Chen (2015) · 2015
Later among the works it cites.
Explaining and generalizing skip-gram through exponential family principal component analysis
Cotterell, R., A. Poliak, B. Van Durme, and J. Eisner (2017) · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…