Fetching the paper…

Reading the bibliography…

2019

Towards Task and Architecture-Independent Generalization Gap Predictors

Yak, Scott, Gonzalvo, Javier, Mazzawi, Hanna

Original
PDF
Post on X

Understand

Can we use deep learning to predict when deep learning works? Our results suggest the affirmative.

We created a dataset by training 13,500 neural networks with different architectures, on different variations of spiral datasets, and using different optimization parameters.
We used this dataset to train task-independent and architecture-independent generalization gap predictors for those neural networks.
We extend Jiang et al.

Towards Task and Architecture-Independent Generalization Gap Predictors · Around

Built on

Variations of the two-spiral task
Chalup, S. and Wiklendt, L · 2007
Earlier work this paper cites.
Learning transferable architectures for scalable image recognition
Original
Zoph, B., Vasudevan, V., Shlens, J., and Le, Q. V · 2007
Earlier work this paper cites.
Understanding deep learning requires rethinking generalization
Original
Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O · 2016
Earlier work this paper cites.
Spectrally-normalized margin bounds for neural networks
Original
Bartlett, P. L., Foster, D. J., and Telgarsky, M · 2017
Earlier work this paper cites.

Similar

Exploring generalization in deep learning
Original
Neyshabur, B., Bhojanapalli, S., McAllester, D., and Srebro, N · 2017
Cited alongside, same era.
Robust large margin deep neural networks
Sokolić, J., Giryes, R., Sapiro, G., and Rodrigues, M. R · 2017
Cited alongside, same era.
Learning and generalization in overparameterized neural networks, going beyond two layers
Original
Allen-Zhu, Z., Li, Y., and Liang, Y · 2018
Cited alongside, same era.
Stronger generalization bounds for deep nets via a compression approach
Original
Arora, S., Ge, R., Neyshabur, B., and Zhang, Y · 2018
Cited alongside, same era.

Then

Large margin deep networks for classification
Elsayed, G., Krishnan, D., Mobahi, H., Regan, K., and Bengio, S · 2018
Later among the works it cites.
Predicting the generalization gap in deep networks with margin distributions
Original
Jiang, Y., Krishnan, D., Mobahi, H., and Bengio, S · 2018
Later among the works it cites.
Towards understanding the role of over-parametrization in generalization of neural networks
Original
Neyshabur, B., Li, Z., Bhojanapalli, S., LeCun, Y., and Srebro, N · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…