Fetching the paper…

Reading the bibliography…

2013

Alternating Minimization for Mixed Linear Regression

Yi, Xinyang, Caramanis, Constantine, Sanghavi, Sujay

Original
PDF
Post on X

Understand

Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one.

It is a classic problem, and the natural and empirically most popular approach to its solution has been the EM algorithm.
As in other settings, this is prone to bad local minima; however, each iteration is very fast (alternating between guessing labels, and solving with those labels).
In this paper we provide a new initialization procedure for EM, based on finding the leading two eigenvectors of an appropriate matrix.

Alternating Minimization for Mixed Linear Regression · Around

Built on

Computers and Intractability: A Guide to the Theory of NP-Completeness
M. Garey and D. Johnson · 1979
Earlier work this paper cites.
On the convergence properties of the em algorithm
C. Wu · 1983
Earlier work this paper cites.
Estimates of use and costs of behavioural health care: a comparison of standard and finite mixture models
P. Deb and A. M. Holmes · 2000
Earlier work this paper cites.
Modeling with mixtures of linear regressions
K. Viele and B. Tong · 2002
Earlier work this paper cites.

Similar

Generalized principal component analysis (gpca)
R. Vidal, Y. Ma, and S. Sastry · 2003
Cited alongside, same era.
Applications of finite mixtures of regression models
B. Grün, F. Leisch, et al · 2007
Cited alongside, same era.
ℓ 1-penalization for mixture regression models
N. Stadler, P. Buhlmann, and S. Geer · 2010
Cited alongside, same era.
Introduction to the non-asymptotic analysis of random matrices
R. Vershynin · 2010
Cited alongside, same era.

Then

Tensor decompositions for learning latent variable models
Original
A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky · 2012
Later among the works it cites.
Sparse subspace clustering: Algorithm, theory, and applications
Original
E. Elhamifar and R. Vidal · 2012
Later among the works it cites.
Learning gaussian mixture models: Moment methods and spectral decompositions
Original
D. Hsu and S. M. Kakade · 2012
Later among the works it cites.
Spectral experts for estimating mixtures of linear regressions
A. Chaganty and P. Liang · 2013
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…