2018

SpectralNet: Spectral Clustering using Deep Neural Networks

Shaham, Uri, Stanton, Kelly, Li, Henry et al.

Understand

Spectral clustering is a leading and popular technique in unsupervised data analysis.

Two of its major limitations are scalability and generalization of the spectral embedding (i.e., out-of-sample-extension).
In this paper we introduce a deep learning approach to spectral clustering that overcomes the above shortcomings.
Our network, which we call SpectralNet, learns a map that embeds input data points into the eigenspace of their associated graph Laplacian matrix and subsequently clusters them.

Built on

Algorithms for the assignment and transportation problems
James Munkres · 1957
Earlier work this paper cites.
Normalized cuts and image segmentation
Jianbo Shi and Jitendra Malik · 2000
Earlier work this paper cites.
On spectral clustering: Analysis and an algorithm
Andrew Y Ng, Michael I Jordan, and Yair Weiss · 2002
Earlier work this paper cites.
Out-of-sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering
Yoshua Bengio, Jean-françcois Paiement, Pascal Vincent, Olivier Delalleau, Nicolas L Roux, and Marie Ouimet · 2004
Earlier work this paper cites.
Spectral grouping using the nystrom method
Charless Fowlkes, Serge Belongie, Fan Chung, and Jitendra Malik · 2004
Earlier work this paper cites.
Dimensionality reduction by learning an invariant mapping
Raia Hadsell, Sumit Chopra, and Yann LeCun · 2006
Earlier work this paper cites.
Diffusion maps, spectral clustering and eigenfunctions of fokker-planck operators
Boaz Nadler, Stephane Lafon, Ioannis Kevrekidis, and Ronald R Coifman · 2006
Earlier work this paper cites.
A tutorial on spectral clustering
Ulrike Von Luxburg · 2007
Earlier work this paper cites.
Multiway spectral clustering with out-of-sample extensions through weighted kernel pca
Carlos Alzate and Johan AK Suykens · 2010
Earlier work this paper cites.

Similar

Locally consistent concept factorization for document clustering
Deng Cai, Xiaofei He, and Jiawei Han · 2011
Cited alongside, same era.
Understanding machine learning: From theory to algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Cited alongside, same era.
Learning deep representations for graph clustering
Fei Tian, Bin Gao, Qing Cui, Enhong Chen, and Tie-Yan Liu · 2014
Cited alongside, same era.
Deep learning with nonparametric clustering
Original
Gang Chen · 2015
Cited alongside, same era.
A stochastic pca and svd algorithm with an exponential convergence rate
Ohad Shamir · 2015
Cited alongside, same era.
Deep unsupervised clustering with gaussian mixture variational autoencoders
Original
Nat Dilokthanakul, Pedro AM Mediano, Marta Garnelo, Matthew CH Lee, Hugh Salimbeni, Kai Arulkumaran, and Murray Shanahan · 2016
Cited alongside, same era.
Deep clustering: Discriminative embeddings for segmentation and separation
John R Hershey, Zhuo Chen, Jonathan Le Roux, and Shinji Watanabe · 2016
Cited alongside, same era.
Unsupervised deep embedding for clustering analysis
Junyuan Xie, Ross Girshick, and Ali Farhadi · 2016
Cited alongside, same era.
Joint unsupervised learning of deep representations and image clusters
Jianwei Yang, Devi Parikh, and Dhruv Batra · 2016
Cited alongside, same era.
Diffusion maps
Ronald R Coifman and Stéphane Lafon
Cited in the paper.
Geometric harmonics: a novel tool for multiscale out-of-sample extension of empirical functions
Ronald R Coifman and Stéphane Lafon
Cited in the paper.

Then

Syncspeccnn: Synchronized spectral cnn for 3d shape segmentation
Original
Li Yi, Hao Su, Xingwen Guo, and Leonidas Guibas · 2016
Later among the works it cites.
Variational deep embedding: A generative approach to clustering
Original
Yin Zheng, Huachun Tan, Bangsheng Tang, Hanning Zhou, et al · 2016
Later among the works it cites.
Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization
Original
Kamran Ghasedi Dizaji, Amirhossein Herandi, and Heng Huang · 2017
Later among the works it cites.
Mini-batch spectral clustering
Yufei Han and Maurizio Filippone · 2017
Later among the works it cites.
Learning discrete representations via information maximizing self augmented training
Original
Weihua Hu, Takeru Miyato, Seiya Tokui, Eiichi Matsumoto, and Masashi Sugiyama · 2017
Later among the works it cites.
Deep spectral clustering learning
Marc T Law, Raquel Urtasun, and Richard S Zemel · 2017
Later among the works it cites.
Diffusion nets
Gal Mishne, Uri Shaham, Alexander Cloninger, and Israel Cohen · 2017
Later among the works it cites.
Towards k-means-friendly spaces: Simultaneous deep learning and clustering
Bo Yang, Xiao Fu, Nicholas D Sidiropoulos, and Mingyi Hong · 2017
Later among the works it cites.
Learning by coincidence: Siamese networks and common variable learning
Uri Shaham and Roy R Lederman · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…