Two Step CCA: A new spectral method for estimating vector models of words

Understand

Unlabeled data is often used to learn representations which can be used to supplement baseline features in a supervised learner.

• For example, for text applications where the words lie in a very high dimensional space (the size of the vocabulary), one can learn a low rank "dictionary" by an eigen-decomposition of the word co-occurrence matrix (e.g.
• using PCA or CCA).
• In this paper, we present a new spectral method based on CCA to learn an eigenword dictionary.