2018

Fast Dynamic Routing Based on Weighted Kernel Density Estimation

Zhang, Suofei, Zhao, Wei, Wu, Xiaofu et al.

Understand

Capsules as well as dynamic routing between them are most recently proposed structures for deep neural networks.

A capsule groups data into vectors or matrices as poses rather than conventional scalars to represent specific properties of target instance.
Besides of pose, a capsule should be attached with a probability (often denoted as activation) for its presence.
The dynamic routing helps capsules achieve more generalization capacity with many fewer model parameters.

Maximum likelihood from incomplete data via the em algorithm
Dempster, A.P., Laird, N.M., Rubin, D.B.: · 1977
Earlier work this paper cites.
Kernel smoothing
Wand, M.P., Jones, M.C.: · 1994
Earlier work this paper cites.
Gradient-based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: · 1998
Earlier work this paper cites.
Real-time tracking of non-rigid objects using mean shift
Comaniciu, D., Ramesh, V., Meer, P.: · 2000
Earlier work this paper cites.
Mean shift: A robust approach toward feature space analysis
Comaniciu, D., Meer, P.: · 2002
Earlier work this paper cites.
Learning methods for generic object recognition with invariance to pose and lighting
LeCun, Y., Huang, F.J., Bottou, L.: · 2004
Earlier work this paper cites.
Gaussian mean-shift is an em algorithm
Carreira-Perpinan, M.A.: · 2007
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G.: · 2009
Earlier work this paper cites.

Transforming auto-encoders
Hinton, G.E., Krizhevsky, A., Wang, S.D.: · 2011
Cited alongside, same era.
An analysis of single-layer networks in unsupervised feature learning
Coates, A., Ng, A., Lee, H.: · 2011
Cited alongside, same era.
Imagenet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., Hinton, G.E.: · 2012
Cited alongside, same era.
Learning mixtures by simplifying kernel density estimators
Schwander, O., Nielsen, F.: · 2013
Cited alongside, same era.
Large-scale video classification with convolutional neural networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: · 2014
Cited alongside, same era.
Deep learning
LeCun, Y., Bengio, Y., Hinton, G.: · 2015
Cited alongside, same era.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: · 2015
Cited alongside, same era.
Coordinate descent algorithms
Wright, S.J.: · 2015
Cited alongside, same era.

Understanding intermediate layers using linear classifier probes
Original
Alain, G., Bengio, Y.: · 2016
Later among the works it cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., Sun, J.: · 2016
Later among the works it cites.
Tensorflow: Large-scale machine learning on heterogeneous distributed systems
Original
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., et al.: · 2016
Later among the works it cites.
Dynamic routing between capsules
Sabour, S., Frosst, N., Hinton, G.E.: · 2017
Later among the works it cites.
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms (2017)
Xiao, H., Rasul, K., Vollgraf, R.: · 2017
Later among the works it cites.
Matrix capsules with em routing
Hinton, G.E., Sabour, S., Frosst, N.: · 2018
Closest in time.
Investigating capsule networks with dynamic routing for text classification
Wei, Z., Jianbo, Y., Min, Y., Zeyang, L., Suofei, Z., Zhou, Z.: · 2018
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…