Fetching the paper…
Reading the bibliography…
How sensitive should machine learning models be to input changes? We tackle the question of model smoothness and show that it is a useful inductive bias which aids generalization, adversarial robustness, generative modeling and reinforcement learning.
For valid generalization the size of the weights is more important than the size of the network
Peter L Bartlett · 1997
Earlier work this paper cites.
Object recognition with gradient-based learning
Yann LeCun, Patrick Haffner, Léon Bottou, and Yoshua Bengio · 1999
Earlier work this paper cites.
Occam’s razor
Carl Edward Rasmussen and Zoubin Ghahramani · 2001
Earlier work this paper cites.
Optimal Transport: Old and New
C. Villani · 2008
Earlier work this paper cites.
Estimating divergence functionals and the likelihood ratio by convex risk minimization
XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan · 2010
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
The nature of statistical learning theory
Vladimir Vapnik · 2013
Earlier work this paper cites.
Intriguing properties of neural networks
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus · 2013
Earlier work this paper cites.
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · 2014
Earlier work this paper cites.
Generative adversarial nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio · 2014
Earlier work this paper cites.
Auto-encoding variational bayes
Diederik P Kingma and Max Welling · 2014
Earlier work this paper cites.
Stochastic backpropagation and approximate inference in deep generative models
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra · 2014
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy · 2015
Earlier work this paper cites.
Variational inference with normalizing flows
Danilo Rezende and Shakir Mohamed · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Train faster, generalize better: Stability of stochastic gradient descent
Moritz Hardt, Ben Recht, and Yoram Singer · 2016
Earlier work this paper cites.
The limitations of deep learning in adversarial settings
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami · 2016
Earlier work this paper cites.
Adversarial machine learning at scale
Alexey Kurakin, Ian Goodfellow, and Samy Bengio · 2016
Earlier work this paper cites.
f-gan: Training generative neural samplers using variational divergence minimization
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka · 2016
Earlier work this paper cites.
Spectral norm regularization for improving the generalizability of deep learning
Yuichi Yoshida and Takeru Miyato · 2017
Earlier work this paper cites.
Parseval networks: improving robustness to adversarial examples
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier · 2017
Earlier work this paper cites.
Robust large margin deep neural networks
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues · 2017
Cited alongside, same era.
Improved training of wasserstein gans
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville · 2017
Cited alongside, same era.
On convergence and stability of gans
Naveen Kodali, Jacob Abernethy, James Hays, and Zsolt Kira · 2017
Cited alongside, same era.
Simple and scalable predictive uncertainty estimation using deep ensembles
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell · 2017
Cited alongside, same era.
Universal adversarial perturbations
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard · 2017
Cited alongside, same era.
Wasserstein generative adversarial networks
Martin Arjovsky, Soumith Chintala, and Léon Bottou · 2017
Implicit quantile networks for distributional reinforcement learning
Will Dabney, Georg Ostrovski, David Silver, and Remi Munos · 2018
Later among the works it cites.
Excessive invariance causes adversarial vulnerability
Joern-Henrik Jacobsen, Jens Behrmann, Richard Zemel, and Matthias Bethge · 2018
Later among the works it cites.
Lipschitz regularity of deep neural networks: analysis and efficient estimation
Aladin Virmaux and Kevin Scaman · 2018
Later among the works it cites.
Deep double descent: Where bigger models and more data hurt
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever · 2019
Later among the works it cites.
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal · 2019
Later among the works it cites.
Training language gans from scratch
Cyprien de Masson d’Autume, Shakir Mohamed, Mihaela Rosca, and Jack Rae · 2019
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
A distributional perspective on reinforcement learning
Marc G Bellemare, Will Dabney, and Rémi Munos · 2017
Cited alongside, same era.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter · 2018
Cited alongside, same era.
Spectral normalization for generative adversarial networks
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida · 2018
Cited alongside, same era.
Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama · 2018
Cited alongside, same era.
Peter L Bartlett, Steven N Evans, and Philip M Long · 2018
Cited alongside, same era.
Many paths to equilibrium: Gans do not need to decrease a divergence at every step
William Fedus, Mihaela Rosca, Balaji Lakshminarayanan, Andrew M Dai, Shakir Mohamed, and Ian Goodfellow · 2018
Cited alongside, same era.
Later among the works it cites.
Invertible residual networks
Jens Behrmann, Will Grathwohl, Ricky TQ Chen, David Duvenaud, and Jörn-Henrik Jacobsen · 2019
Later among the works it cites.
Implicit competitive regularization in gans
Florian Schäfer, Hongkai Zheng, and Anima Anandkumar · 2019
Later among the works it cites.
Lipschitz generative adversarial nets
Zhiming Zhou, Jiadong Liang, Yuxuan Song, Lantao Yu, Hongwei Wang, Weinan Zhang, Yong Yu, and Zhihua Zhang · 2019
Later among the works it cites.
Efficient and accurate estimation of lipschitz constants for deep neural networks
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George Pappas · 2019
Later among the works it cites.
A large-scale study on regularization and normalization in gans
Karol Kurach, Mario Lučić, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly · 2019
Later among the works it cites.
Lipschitz certificates for neural network structures driven by averaged activation operators
Patrick L Combettes and Jean-Christophe Pesquet · 2019
Later among the works it cites.
Regularisation of neural networks by enforcing lipschitz continuity
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J Cree · 2020
Closest in time.
The lipschitz constant of self-attention
Hyunjik Kim, George Papamakarios, and Andriy Mnih · 2020
Closest in time.
Distribution augmentation for generative modeling
Heewoo Jun, Rewon Child, Mark Chen, John Schulman, Aditya Ramesh, Alec Radford, and Ilya Sutskever · 2020
Closest in time.
Simple and scalable epistemic uncertainty estimation using a single deep deterministic neural network
Joost van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal · 2020
Closest in time.
Simple and principled uncertainty estimation with deterministic deep learning via distance awareness
Jeremiah Zhe Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss, and Balaji Lakshminarayanan · 2020
Closest in time.
Nvae: A deep hierarchical variational autoencoder
Arash Vahdat and Jan Kautz · 2020
Closest in time.
Kale: When energy-based learning meets adversarial training
Michael Arbel, Liang Zhou, and Arthur Gretton · 2020
Closest in time.
On mutual information maximization for representation learning
Michael Tobias Tschannen, Josip Djolonga, Paul Kishan Rubenstein, Sylvain Gelly, and Mario Lučić · 2020
Closest in time.
How to learn a useful critic? model-based action-gradient-estimator policy optimization
Pierluca D’Oro and Wojciech Jaśkowski · 2020
Closest in time.
How to train your neural ode
Chris Finlay, Jörn-Henrik Jacobsen, Levon Nurbekyan, and Adam M Oberman · 2020
Closest in time.
A general approach to fairness with optimal transport
Chiappa Silvia, Jiang Ray, Stepleton Tom, Pacchiano Aldo, Jiang Heinrich, and Aslanides John · 2020
Closest in time.
Wasserstein fair classification
Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa · 2020
Closest in time.