On improving deep learning generalization with adaptive sparse connectivity

Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization

Original

Mostafa, H. and Wang, X · 1902

Earlier work this paper cites.

Optimal brain damage

LeCun, Y., Denker, J. S., and Solla, S. A · 1990

Earlier work this paper cites.

Second order derivatives for network pruning: Optimal brain surgeon

Hassibi, B. and Stork, D. G · 1993

Earlier work this paper cites.

In search of the real inductive bias: On the role of implicit regularization in deep learning

Original

Neyshabur, B., Tomioka, R., and Srebro, N · 2014

Earlier work this paper cites.

Dropout: a simple way to prevent neural networks from overfitting

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R · 2014

Earlier work this paper cites.

Batch normalization: Accelerating deep network training by reducing internal covariate shift

Original

Ioffe, S. and Szegedy, C · 2015

Earlier work this paper cites.

On large-batch training for deep learning: Generalization gap and sharp minima

Original

Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M., and Tang, P. T. P · 2016

Cited alongside, same era.

Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data

Original

Dziugaite, G. K. and Roy, D. M · 2017

Cited alongside, same era.

Ese: Efficient speech recognition engine with sparse lstm on fpga

Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., et al · 2017

Cited alongside, same era.

In-datacenter performance analysis of a tensor processing unit

Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al · 2017

Cited alongside, same era.

Exploring sparsity in recurrent neural networks

Original

Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science

Mocanu, D. C., Mocanu, E., Stone, P., Nguyen, P. H., Gibescu, M., and Liotta, A · 2018

Later among the works it cites.

Improvement of generalization ability of deep cnn via implicit regularization in two-stage training process

Zheng, Q., Yang, M., Yang, J., Zhang, Q., and Zhang, X · 2018

Later among the works it cites.

Multi-objective evolutionary federated learning

Original

Zhu, H. and Jin, Y · 2018

Later among the works it cites.

Sparse evolutionary deep learning with over one million artificial neurons on commodity hardware

Original

Liu, S., Mocanu, D. C., Matavalam, A. R. R., Pei, Y., and Pechenizkiy, M · 2019

Closest in time.

Low-memory neural network training: A technical report

Original