2019

Meta-Learning to Cluster

Jiang, Yibo, Verma, Nakul

Understand

Clustering is one of the most fundamental and wide-spread techniques in exploratory data analysis.

Yet, the basic approach to clustering has not really changed: a practitioner hand-picks a task-specific clustering loss to optimize and fit the given data to reveal the underlying cluster structure.
Some types of losses---such as k-means, or its non-linear version: kernelized k-means (centroid based), and DBSCAN (density based)---are popular choices due to their good empirical performance on a range of applications.
Although every so often the clustering output using these standard losses fails to reveal the underlying structure, and the practitioner has to custom-design their own variation.

Built on

Object recognition and localization via pose clustering
G. Stockman · 1987
Earlier work this paper cites.
A density-based algorithm for discovering clusters in large spatial databases with noise
M. Ester, H. Kriegel, J. Sander, and X. Xu · 1996
Earlier work this paper cites.
Long short-term memory
S. Hochreiter and J. Schmidhuber · 1997
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
An impossibility theorem for clustering
J. M. Kleinberg · 2002
Earlier work this paper cites.
Document clustering based on non-negative matrix factorization
W. Xu, X. Liu, and Y. Gong · 2003
Earlier work this paper cites.
Hierarchical clustering of www image search results using visual, textual and link information
D. Cai, X. He, Z. Li, W.-Y. Ma, and J.-R. Wen · 2004
Earlier work this paper cites.
Reducing the dimensionality of data with neural networks
G. E. Hinton and R. R. Salakhutdinov · 2006
Earlier work this paper cites.
A recommender system using ga k-means clustering in an online shopping market
K. Kim and H. Ahn · 2008
Earlier work this paper cites.
Enabling scalable spectral clustering for image segmentation
F. Tung, A. Wong, and D. A. Clausi · 2010
Earlier work this paper cites.
One shot learning of simple visual concepts
B. M. Lake, R. Salakhutdinov, J. Gross, and J. B. Tenenbaum · 2011
Earlier work this paper cites.

Similar

Clustering algorithm recommendation: A meta-learning approach
D. G. Ferrari and L. N. de Castro · 2012
Cited alongside, same era.
The development of intuitive knowledge classifier and the modeling of domain dependent data
H. T. Kahraman, S. Sagiroglu, and I. Colak · 2012
Cited alongside, same era.
Openml: Networked science in machine learning
J. Vanschoren, J. N. van Rijn, B. Bischl, and L. Torgo · 2013
Cited alongside, same era.
Clustering algorithm selection by meta-learning systems: A new distance-based problem characterization and ranking combination methods
D. G. Ferrari and L. N. de Castro · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Original
D. P. Kingma and J. Ba · 2014
Cited alongside, same era.
A comprehensive survey of clustering algorithms
D. Xu and Y. Tian · 2015
Cited alongside, same era.
Learning to navigate in complex environments
Original
P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard, A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuoglu, D. Kumaran, and R. Hadsell · 2016
Cited alongside, same era.
Meta-learning with memory-augmented neural networks
A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. P. Lillicrap · 2016
Cited alongside, same era.
Unsupervised deep embedding for clustering analysis
J. Xie, R. B. Girshick, and A. Farhadi · 2016
Cited alongside, same era.
UCI machine learning repository, 2017
D. Dheeru and E. Karra Taniskidou · 2017
Cited alongside, same era.
Model-agnostic meta-learning for fast adaptation of deep networks
C. Finn, P. Abbeel, and S. Levine · 2017
Cited alongside, same era.
Unsupervised deep embedding for clustering analysis
J. Xie, R. Girshick, and A. Farhadi
Cited in the paper.

Then

Learning discrete representations via information maximizing self-augmented training
W. Hu, T. Miyato, S. Tokui, E. Matsumoto, and M. Sugiyama · 2017
Later among the works it cites.
A simple neural attentive meta-learner
N. Mishra, M. Rohaninejad, X. Chen, and P. Abbeel · 2017
Later among the works it cites.
Prototypical networks for few-shot learning
J. Snell, K. Swersky, and R. S. Zemel · 2017
Later among the works it cites.
Learning to reinforcement learn
Original
J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Kumaran, and M. Botvinick · 2017
Later among the works it cites.
Towards k-means-friendly spaces: Simultaneous deep learning and clustering
Original
B. Yang, X. Fu, N. D. Sidiropoulos, and M. Hong · 2017
Later among the works it cites.
Clustering with deep learning: Taxonomy and new methods
Original
E. Aljalbout, V. Golkov, Y. Siddiqui, and D. Cremers · 2018
Later among the works it cites.
Supervising unsupervised learning
V. Garg · 2018
Later among the works it cites.
Unsupervised learning via meta-learning
Original
K. Hsu, S. Levine, and C. Finn · 2018
Later among the works it cites.
On first-order meta-learning algorithms
Original
A. Nichol, J. Achiam, and J. Schulman · 2018
Later among the works it cites.
Prefrontal cortex as a meta-reinforcement learning system
J. X. Wang, Z. Kurth-Nelson, D. Kumaran, D. Tirumala, H. Soyer, J. Z. Leibo, D. Hassabis, and M. M. Botvinick · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…