Training behavior of deep neural network in frequency domain

Zhang, Y., Xu, Z.Q.J., Luo, T., Ma, Z.: Explicitizing an implicit bias of the frequency principle in two-layer neural networks. arXiv:1905.10264 [cs, stat] (May 2019), http://arxiv.org/abs/1905.10264 , arXiv: 1905.10264

Original

1905

Earlier work this paper cites.

Zhang, Y., Xu, Z.Q.J., Luo, T., Ma, Z.: A type of generalization error induced by initialization in deep neural networks. arXiv:1905.07777 [cs, stat] (May 2019), http://arxiv.org/abs/1905.07777 , arXiv: 1905.07777

Original

1905

Earlier work this paper cites.

Shannon, C.E.: Communication in the presence of noise. Proceedings of the IRE 37

1949

Earlier work this paper cites.

Yen, J.: On nonuniform sampling of bandwidth-limited signals. IRE Transactions on circuit theory 3

1956

Earlier work this paper cites.

Percival, D.B., Walden, A.T.: Spectral analysis for physical applications. cambridge university press (1993)

1993

Earlier work this paper cites.

Mishali, M., Eldar, Y.C.: Blind multiband signal reconstruction: Compressed sensing for analog signals. IEEE Transactions on Signal Processing 57

2009

Earlier work this paper cites.

Hackbusch, W.: Multi-grid methods and applications, vol. 4. Springer Science and Business Media (2013)

2013

Earlier work this paper cites.

Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

Original

2014

Earlier work this paper cites.

Hardt, M., Recht, B., Singer, Y.: Train faster, generalize better: Stability of stochastic gradient descent. arXiv preprint arXiv:1509.01240 (2015)

Original

2015

Earlier work this paper cites.

LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. nature 521

2015

Earlier work this paper cites.

Alain, G., Bengio, Y.: Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644 (2016)

Original

2016

Cited alongside, same era.

Lin, J., Camoriano, R., Rosasco, L.: Generalization properties and implicit regularization for multiple passes sgm. In: International Conference on Machine Learning. pp. 2340–2348 (2016)

2016

Cited alongside, same era.

Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)

Original

2016

Cited alongside, same era.

Arpit, D., Jastrzebski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M.S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al.: A closer look at memorization in deep networks. arXiv preprint arXiv:1706.05394 (2017)

Original

2017

Cited alongside, same era.

Zheng, G., Sang, J., Xu, C.: Understanding deep learning generalization by maximum entropy. arXiv preprint arXiv:1711.07758 (2017)

Original

2017

Later among the works it cites.

Rahaman, N., Arpit, D., Baratin, A., Draxler, F., Lin, M., Hamprecht, F.A., Bengio, Y., Courville, A.: On the spectral bias of deep neural networks. arXiv preprint arXiv:1806.08734 (2018)

Original

2018

Closest in time.

Saxe, A.M., Bansal, Y., Dapello, J., Advani, M.: On the information bottleneck theory of deep learning. International Conference on Learning Representations (2018)

2018

Closest in time.

Xu, Z.Q.J.: Frequency principle in deep learning with general loss functions and its potential application. arXiv preprint arXiv:1811.10146 (2018)

Original

2018

Closest in time.

Xu, Z.J.: Understanding training and generalization in deep learning by fourier analysis. arXiv preprint arXiv:1808.04295 (2018)