Bringing Giant Neural Networks Down to Earth with Unlabeled Data

R. Burbidge, M. Trotter, B. Buxton, and S. Holden, “Drug design by machine learning: support vector machines for pharmaceutical data analysis,” Computers & chemistry , vol. 26, no. 1, pp. 5–14, 2001

2001

Earlier work this paper cites.

V. Koltchinskii, D. Panchenko et al. , “Empirical margin distributions and bounding the generalization error of combined classifiers,” The Annals of Statistics , vol. 30, no. 1, pp. 1–50, 2002

2002

Earlier work this paper cites.

J. Goldberger, S. Gordon, and H. Greenspan, “An efficient image similarity measure based on approximations of kl-divergence between two gaussian mixtures,” in null . IEEE, 2003, p. 487

2003

Earlier work this paper cites.

J. Quiñonero-Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence, “Covariate shift and local learning by distribution matching,” 2008

2008

Earlier work this paper cites.

A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM journal on imaging sciences , vol. 2, no. 1, pp. 183–202, 2009

2009

Earlier work this paper cites.

A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009

2009

Earlier work this paper cites.

V. Vanhoucke, A. Senior, and M. Z. Mao, “Improving the speed of neural networks on cpus,” 2011

2011

Earlier work this paper cites.

A. Coates, A. Y. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” in AISTATS , ser. JMLR Proceedings, vol. 15. JMLR.org, 2011, pp. 215–223

2011

Earlier work this paper cites.

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems , 2012, pp. 1097–1105

2012

Earlier work this paper cites.

M. Denil, B. Shakibi, L. Dinh, M. Ranzato, and N. de Freitas, “Predicting parameters in deep learning,” in Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States. , 2013, pp. 2148–2156. [Online]. Available: http://papers.nips.cc/paper/5025-predicting-parameters-in-deep-learning

2013

Earlier work this paper cites.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

Original

2014

Earlier work this paper cites.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems , 2014, pp. 2672–2680

2014

Earlier work this paper cites.

Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” arXiv preprint arXiv:1409.7495 , 2014

Original

2014

Earlier work this paper cites.

T. Lin, M. Maire, S. J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: common objects in context,” in ECCV (5) , ser. Lecture Notes in Computer Science, vol. 8693. Springer, 2014, pp. 740–755

2014

Earlier work this paper cites.

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 1–9

2015

Earlier work this paper cites.

S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” CoRR , vol. abs/1510.00149, 2015

Original

2015

Earlier work this paper cites.

G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

Original

2015

Earlier work this paper cites.

M. Courbariaux, Y. Bengio, and J.-P. David, “Binaryconnect: Training deep neural networks with binary weights during propagations,” in Advances in neural information processing systems , 2015, pp. 3123–3131

2015

Earlier work this paper cites.

S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” in Advances in neural information processing systems , 2015, pp. 1135–1143

2015

Earlier work this paper cites.

S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149 , 2015

Original

2015

Earlier work this paper cites.

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167 , 2015

Original

2015

Earlier work this paper cites.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV) , vol. 115, no. 3, pp. 211–252, 2015

2015

Earlier work this paper cites.

D. J. Rezende, S. Mohamed, I. Danihelka, K. Gregor, and D. Wierstra, “One-shot generalization in deep generative models,” in ICML , ser. JMLR Workshop and Conference Proceedings, vol. 48. JMLR.org, 2016, pp. 1521–1529

2016

Earlier work this paper cites.

L. Maaloe, C. K. Sonderby, S. K. Sonderby, and O. Winther, “Auxiliary deep generative models,” in ICML , ser. JMLR Workshop and Conference Proceedings, vol. 48. JMLR.org, 2016, pp. 1445–1453

2016

Earlier work this paper cites.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

2016

Cited alongside, same era.

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Binarized neural networks,” in Advances in neural information processing systems , 2016, pp. 4107–4115

2016

Cited alongside, same era.

X. Zhang, J. Zou, K. He, and J. Sun, “Accelerating very deep convolutional networks for classification and detection,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 38, no. 10, pp. 1943–1955, 2016

2016

Cited alongside, same era.

J. M. Alvarez and M. Salzmann, “Learning the number of neurons in deep networks,” in Advances in Neural Information Processing Systems , 2016, pp. 2270–2278

2016

Cited alongside, same era.

Y. He, X. Zhang, and J. Sun, “Channel pruning for accelerating very deep neural networks,” in Proceedings of the IEEE International Conference on Computer Vision , 2017, pp. 1389–1397

2017

Later among the works it cites.

C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. P. Aitken, A. Tejani, J. Totz, Z. Wang et al. , “Photo-realistic single image super-resolution using a generative adversarial network.” in CVPR , vol. 2, no. 3, 2017, p. 4

2017

Later among the works it cites.

E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarial discriminative domain adaptation,” in Computer Vision and Pattern Recognition (CVPR) , vol. 1, no. 2, 2017, p. 4

2017

Later among the works it cites.

K. Kawaguchi, L. P. Kaelbling, and Y. Bengio, “Generalization in deep learning,” arXiv preprint arXiv:1710.05468 , 2017

Original

2017

Later among the works it cites.