Fetching the paper…
Reading the bibliography…
We explore the link between deep ensembles and Gaussian processes (GPs) through the lens of the Neural Tangent Kernel (NTK): a recent development in understanding the training dynamics of wide neural networks (NNs).
Greg Yang · 1902
Earlier work this paper cites.
Greg Yang · 1910
Earlier work this paper cites.
Goodness of Prediction Fit
James Aitchison · 1975
Earlier work this paper cites.
Probability and Measure
Patrick Billingsley · 1986
Earlier work this paper cites.
Constrained Realizations of Gaussian Fields: A Simple Algorithm
Yehuda Hoffman and Erez Ribak · 1991
Earlier work this paper cites.
Probability with Martingales
David Williams · 1991
Earlier work this paper cites.
Bayesian Methods for Adaptive Models
David JC MacKay · 1992
Earlier work this paper cites.
Priors for Infinite Networks
Radford M Neal · 1996
Earlier work this paper cites.
Computing with Infinite Networks
Christopher KI Williams · 1997
Earlier work this paper cites.
Gaussian Processes in Machine Learning
Carl Edward Rasmussen · 2003
Earlier work this paper cites.
Bayesian Learning via Stochastic Gradient Langevin Dynamics
Max Welling and Yee W Teh · 2011
Earlier work this paper cites.
Practical Variational Inference for Neural Networks
Alex Graves · 2011
Earlier work this paper cites.
Bayesian Learning for Neural Networks , volume 118
Radford M Neal · 2012
Earlier work this paper cites.
Efficient BackProp
Yann A LeCun, Léon Bottou, Genevieve B Orr, and Klaus-Robert Müller · 2012
Earlier work this paper cites.
Gaussian Processes for Big Data
James Hensman, Nicolo Fusi, and Neil D Lawrence · 2013
Earlier work this paper cites.
Adam: A Method for Stochastic Optimization
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Weight Uncertainty in Neural Networks
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra · 2015
Earlier work this paper cites.
Delving Deep into Rectifiers: Surpassing Human-Level Performance on Imagenet Classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
Yarin Gal and Zoubin Ghahramani · 2016
Earlier work this paper cites.
Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
Christos Louizos and Max Welling · 2017
Earlier work this paper cites.
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell · 2017
Cited alongside, same era.
Sample-then-optimize posterior sampling for Bayesian linear models
Alexander G de G Matthews, Jiri Hron, Richard E Turner, and Zoubin Ghahramani · 2017
Cited alongside, same era.
The Reversible Residual Network: Backpropagation Without Storing Activations
Aidan N Gomez, Mengye Ren, Raquel Urtasun, and Roger B Grosse · 2017
Cited alongside, same era.
Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches
Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, and Roger Grosse · 2018
Cited alongside, same era.
Deep Neural Networks as Gaussian Processes
Jaehoon Lee, Jascha Sohl-dickstein, Jeffrey Pennington, Roman Novak, Sam Schoenholz, and Yasaman Bahri · 2018
Cited alongside, same era.
On Lazy Training in Differentiable Programming
Lenaic Chizat, Edouard Oyallon, and Francis Bach · 2019
Later among the works it cites.
Asymptotics of Wide Networks from Feynman Diagrams
Ethan Dyer and Guy Gur-Ari · 2019
Later among the works it cites.
Dynamics of Deep Neural Networks and Neural Tangent Hierarchy
Jiaoyang Huang and Horng-Tzer Yau · 2019
Later among the works it cites.
Mean-field Behaviour of Neural Tangent Kernel for Deep Neural Networks
Soufiane Hayou, Arnaud Doucet, and Judith Rousseau · 2019
Later among the works it cites.
Bayesian Deep Learning and a Probabilistic Perspective of Generalization
Andrew Gordon Wilson and Pavel Izmailov · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Gaussian Process Behaviour in Wide Deep Neural Networks
Alexander G de G Matthews, Mark Rowland, Jiri Hron, Richard E Turner, and Zoubin Ghahramani · 2018
Cited alongside, same era.
Neural Tangent Kernel: Convergence and Generalization in Neural Networks
Arthur Jacot, Franck Gabriel, and Clément Hongler · 2018
Cited alongside, same era.
Randomized Prior Functions for Deep Reinforcement Learning
Ian Osband, John Aslanides, and Albin Cassirer · 2018
Cited alongside, same era.
Uncertainty in Neural Networks: Bayesian Ensembling
Tim Pearce, Mohamed Zaki, Alexandra Brintrup, Nicolas Anastassacos, and Andy Neely · 2018
Cited alongside, same era.
JAX: composable transformations of Python+NumPy programs, 2018
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, and Skye Wanderman-Milne · 2018
Cited alongside, same era.
Functional Variational Bayesian Neural Networks
Shengyang Sun, Guodong Zhang, Jiaxin Shi, and Roger Grosse · 2019
Cited alongside, same era.
Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V Dillon, Balaji Lakshminarayanan, and Jasper Snoek · 2019
Cited alongside, same era.
Closest in time.
Infinite attention: NNGP and NTK for deep attention networks
Jiri Hron, Yasaman Bahri, Jascha Sohl-Dickstein, and Roman Novak · 2020
Closest in time.
Conservative Uncertainty Estimation By Fitting Prior Networks
Kamil Ciosek, Vincent Fortuin, Ryota Tomioka, Katja Hofmann, and Richard Turner · 2020
Closest in time.
Tensor Programs II: Neural Tangent kernel for Any Architecture
Greg Yang · 2020
Closest in time.
On the infinite width limit of neural networks with a standard parameterization
Jascha Sohl-Dickstein, Roman Novak, Samuel S Schoenholz, and Jaehoon Lee · 2020
Closest in time.
Simple and Effective Regularization Methods for Training on Noisily Labeled Data with Generalization Guarantee
Wei Hu, Zhiyuan Li, and Dingli Yu · 2020
Closest in time.
Neural Tangents: Fast and Easy Infinite Neural Networks in Python
Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, and Samuel S. Schoenholz · 2020
Closest in time.
Neural Kernels Without Tangents
Vaishaal Shankar, Alex Fang, Wenshuo Guo, Sara Fridovich-Keil, Ludwig Schmidt, Jonathan Ragan-Kelley, and Benjamin Recht · 2020
Closest in time.
On the marginal likelihood and cross-validation
E Fong and C C Holmes · 2020
Closest in time.
Kernel and Rich Regimes in Overparametrized Models
Blake Woodworth, Suriya Gunasekar, Jason D Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, and Nathan Srebro · 2020
Closest in time.
Finite Depth and Width Corrections to the Neural Tangent Kernel
Boris Hanin and Mihai Nica · 2020
Closest in time.
Neural Ensemble Search for Performant and Calibrated Predictions
Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter, and Yee Whye Teh · 2020
Closest in time.
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng · 2020
Closest in time.
Hydra: Preserving Ensemble Diversity for Model Distillation
Linh Tran, Bastiaan S Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V Dillon, Jasper Snoek, Stephan Mandt, Tim Salimans, Sebastian Nowozin, and Rodolphe Jenatton · 2020
Closest in time.
Uncertainty Estimation Using a Single Deep Deterministic Neural Network
Joost van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal · 2020
Closest in time.
Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
Michael W Dusenberry, Ghassen Jerfel, Yeming Wen, Yi-an Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, and Dustin Tran · 2020
Closest in time.