Understanding training and generalization in deep learning by Fourier analysis

Shannon, C. E. (1949), ‘Communication in the presence of noise’, Proceedings of the IRE

1949

Earlier work this paper cites.

Yen, J. (1956), ‘On nonuniform sampling of bandwidth-limited signals’, IRE Transactions on circuit theory

1956

Earlier work this paper cites.

Cybenko, G. (1989), ‘Approximation by superpositions of a sigmoidal function’, Mathematics of control, signals and systems

1989

Earlier work this paper cites.

Dong, D. W. & Atick, J. J. (1995), ‘Statistics of natural time-varying images’, Network: Computation in Neural Systems

1995

Earlier work this paper cites.

Hochreiter, S. & Schmidhuber, J. (1995), Simplifying neural nets by discovering flat minima, in

1995

Earlier work this paper cites.

LeCun, Y. (1998), ‘The mnist database of handwritten digits’, http://yann. lecun. com/exdb/mnist/

1998

Earlier work this paper cites.

Vapnik, V. N. (1999), ‘An overview of statistical learning theory’, IEEE transactions on neural networks

1999

Earlier work this paper cites.

Bousquet, O. & Elisseeff, A. (2002), ‘Stability and generalization’, Journal of machine learning research

2002

Earlier work this paper cites.

Mishali, M. & Eldar, Y. C. (2009), ‘Blind multiband signal reconstruction: Compressed sensing for analog signals’, IEEE Transactions on Signal Processing

2009

Cited alongside, same era.

Kingma, D. P. & Ba, J. (2014), ‘Adam: A method for stochastic optimization’, arXiv preprint arXiv:1412.6980

Original

2014

Cited alongside, same era.

Hardt, M., Recht, B. & Singer, Y. (2015), ‘Train faster, generalize better: Stability of stochastic gradient descent’, arXiv preprint arXiv:1509.01240

Original

2015

Cited alongside, same era.

LeCun, Y., Bengio, Y. & Hinton, G. (2015), ‘Deep learning’, nature

2015

Cited alongside, same era.

Keskar, N. S., Mudigere, D., Nocedal, J., Smelyanskiy, M. & Tang, P. T. P. (2016), ‘On large-batch training for deep learning: Generalization gap and sharp minima’, arXiv preprint arXiv:1609.04836

Original

Dinh, L., Pascanu, R., Bengio, S. & Bengio, Y. (2017), ‘Sharp minima can generalize for deep nets’, arXiv preprint arXiv:1703.04933

Original

2017

Later among the works it cites.

Kawaguchi, K., Kaelbling, L. P. & Bengio, Y. (2017), ‘Generalization in deep learning’, arXiv preprint arXiv:1710.05468

Original

2017

Later among the works it cites.

Nagarajan, V. & Kolter, J. Z. (2017), Generalization in deep networks: The role of distance from initialization, in

2017

Later among the works it cites.

Wu, L., Zhu, Z. & E, W. (2017), ‘Towards understanding generalization of deep learning: Perspective of loss landscapes’, arXiv preprint arXiv:1706.10239

Original

2017

Later among the works it cites.

Poggio, T., Kawaguchi, K., Liao, Q., Miranda, B., Rosasco, L., Boix, X., Hidary, J. & Mhaskar, H. (2018), Theory of deep learning iii: the non-overfitting puzzle, Technical report, Technical report, CBMM memo 073