Compression and Interpretability of Deep Neural Networks via Tucker Tensor Layer: From First Principles to Tensor Valued Back-Propagation

L. R. Tucker, “Implications of factor analysis of three-way matrices for measurement of change,” in Problems in Measuring Change , C. W. Harris, Ed. Madison WI: University of Wisconsin Press, 1963, pp. 122–137

1963

Earlier work this paper cites.

J. R. Magnus and H. Neudecker, “Matrix Differential Calculus with Applications to Simple, Hadamard, and Kronecker Products,” Journal of Mathematical Psychology , vol. 29, no. 4, pp. 474–492, 1985

1985

Earlier work this paper cites.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature , vol. 323, no. 6088, p. 533, 1986

1986

Earlier work this paper cites.

S. Wold, K. Esbensen, and P. Geladi, “Principal component analysis,” Chemometrics and Intelligent Laboratory Systems , vol. 2, no. 1-3, pp. 37–52, 1987

1987

Earlier work this paper cites.

R. Bro, “PARAFAC. Tutorial and applications,” Chemometrics and Intelligent Laboratory Systems , vol. 38, no. 2, pp. 149–171, 1997

1997

Earlier work this paper cites.

Y. LeCun, “The MNIST database of handwritten digits,” http://yann. lecun. com/exdb/mnist/ , 1998

1998

Earlier work this paper cites.

L. D. Lathauwer, B. D. Moor, and J. Vandewalle, “A multilinear singular value decomposition,” SIAM Journal on Matrix Analysis and Applications , vol. 21, no. 4, pp. 1253–1278, 2000

2000

Earlier work this paper cites.

D. P. Mandic and J. Chambers, Recurrent neural networks for prediction: learning algorithms, architectures and stability . John Wiley & Sons, Inc., 2001

2001

Earlier work this paper cites.

T. Kolda and B. Bader, “Tensor decompositions and applications,” SIAM Review , vol. 51, no. 3, pp. 455–500, 2009

2009

Earlier work this paper cites.

A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009

2009

Cited alongside, same era.

I. V. Oseledets, “Tensor-train decomposition,” SIAM Journal on Scientific Computing , vol. 33, no. 5, pp. 2295–2317, 2011

2011

Cited alongside, same era.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS) , 2012, pp. 1097–1105

2012

Cited alongside, same era.

A. Graves, A.-r. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2013, pp. 6645–6649

2013

Cited alongside, same era.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al. , “Imagenet large scale visual recognition challenge,” International Journal of Computer Vision , vol. 115, no. 3, pp. 211–252, 2015

2015

Later among the works it cites.

A. Cichocki, D. P. Mandic, A. H. Phan, C. F. Caiafa, G. Zhou, Q. Zhao, and L. D. Lathauwer, “Tensor decompositions for signal processing applications,” IEEE Signal Processing Magazine , vol. 32, no. 2, pp. 145–163, 2015

2015

Later among the works it cites.

A. Novikov, D. Podoprikhin, A. Osokin, and D. P. Vetrov, “Tensorizing neural networks,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS) , 2015, pp. 442–450

2015

Later among the works it cites.

Y.-D. Kim, E. Park, S. Yoo, T. Choi, L. Yang, and D. Shin, “Compression of deep convolutional neural networks for fast and low power mobile applications,” arXiv preprint arXiv:1511.06530 , 2015

Original

2015

Later among the works it cites.