Fetching the paper…
Reading the bibliography…
Deep neural nets typically perform end-to-end backpropagation to learn the weights, a procedure that creates synchronization constraints in the weight update step across layers and is not biologically plausible.
The organization of behavior: a neuropsychological theory
D. O. Hebb · 1949
Earlier work this paper cites.
Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex
E. L. Bienenstock, L. N. Cooper, and P. W. Munro · 1982
Earlier work this paper cites.
Simplified neuron model as a principal component analyzer
E. Oja · 1982
Earlier work this paper cites.
Information processing in dynamical systems: Foundations of harmony theory
P. Smolensky · 1986
Earlier work this paper cites.
Synaptic plasticity: taming the beast
L. F. Abbott and S. B. Nelson · 2000
Earlier work this paper cites.
Greedy layer-wise training of deep networks
Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle · 2006
Earlier work this paper cites.
A fast learning algorithm for deep belief nets
G. E. Hinton, S. Osindero, and Y. W. Teh · 2006
Earlier work this paper cites.
Spike timing–dependent plasticity: a hebbian learning rule
N. Caporale and Y. Dan · 2008
Earlier work this paper cites.
Visualizing data using t-sne
L. v. d. Maaten and G. Hinton · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
An analysis of single-layer networks in unsupervised feature learning
A. Coates, A. Ng, and H. Lee · 2011
Earlier work this paper cites.
A practical guide to training restricted boltzmann machines
G. E. Hinton · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
How auto-encoders could provide credit assignment in deep networks via target propagation
Y. Bengio · 2014
Earlier work this paper cites.
One weird trick for parallelizing convolutional neural networks
A. Krizhevsky · 2014
Earlier work this paper cites.
Microsoft COCO: common objects in context
T. Lin, M. Maire, S. J. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick · 2014
Earlier work this paper cites.
Towards biologically plausible deep learning
Y. Bengio, D. Lee, J. Bornschein, and Z. Lin · 2015
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
C. Doersch, A. Gupta, and A. A. Efros · 2015
Earlier work this paper cites.
Training deep nets with sublinear memory cost
T. Chen, B. Xu, C. Zhang, and C. Guestrin · 2016
Earlier work this paper cites.
The cityscapes dataset for semantic urban scene understanding
M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele · 2016
Earlier work this paper cites.
Memory-efficient backpropagation through time
A. Gruslys, R. Munos, I. Danihelka, M. Lanctot, and A. Graves · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Cited alongside, same era.
Sgdr: Stochastic gradient descent with warm restarts
I. Loshchilov and F. Hutter · 2016
Cited alongside, same era.
Direct feedback alignment provides learning in deep neural networks
A. Nøkland · 2016
Cited alongside, same era.
Unsupervised learning of visual representations by solving jigsaw puzzles
M. Noroozi and P. Favaro · 2016
Cited alongside, same era.
Colorful image colorization
R. Zhang, P. Isola, and A. A. Efros · 2016
Cited alongside, same era.
The reversible residual network: Backpropagation without storing activations
A. N. Gomez, M. Ren, R. Urtasun, and R. B. Grosse · 2017
Cited alongside, same era.
MMDetection: Open mmlab detection toolbox and benchmark
K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin · 2019
Later among the works it cites.
Scaling and benchmarking self-supervised visual representation learning
P. Goyal, D. Mahajan, A. Gupta, and I. Misra · 2019
Later among the works it cites.
Momentum contrast for unsupervised visual representation learning
K. He, H. Fan, Y. Wu, S. Xie, and R. B. Girshick · 2019
Later among the works it cites.
Rethinking imagenet pre-training
K. He, R. Girshick, and P. Dollár · 2019
Later among the works it cites.
Bag of tricks for image classification with convolutional neural networks
T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, and M. Li · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Mask R-CNN
K. He, G. Gkioxari, P. Dollár, and R. B. Girshick · 2017
Cited alongside, same era.
Decoupled neural interfaces using synthetic gradients
M. Jaderberg, W. M. Czarnecki, S. Osindero, O. Vinyals, A. Graves, D. Silver, and K. Kavukcuoglu · 2017
Cited alongside, same era.
Representation learning by learning to count
M. Noroozi, H. Pirsiavash, and P. Favaro · 2017
Cited alongside, same era.
Large batch training of convolutional networks
Y. You, I. Gitman, and B. Ginsburg · 2017
Cited alongside, same era.
Greedy layerwise learning can scale to imagenet
E. Belilovsky, M. Eickenberg, and E. Oyallon · 2018
Cited alongside, same era.
Deep clustering for unsupervised learning of visual features
M. Caron, P. Bojanowski, A. Joulin, and M. Douze · 2018
Cited alongside, same era.
Gpipe: Efficient training of giant neural networks using pipeline parallelism
Y. Huang, Y. Cheng, A. Bapna, O. Firat, D. Chen, M. Chen, H. Lee, J. Ngiam, Q. V. Le, Y. Wu, et al · 2019
Later among the works it cites.
Revisiting self-supervised visual representation learning
A. Kolesnikov, X. Zhai, and L. Beyer · 2019
Later among the works it cites.
Selective kernel networks
X. Li, W. Wang, X. Hu, and J. Yang · 2019
Later among the works it cites.
Putting an end to end-to-end: Gradient-isolated learning of representations
S. Löwe, P. O’Connor, and B. S. Veeling · 2019
Later among the works it cites.
Meta-learning update rules for unsupervised representation learning
L. Metz, N. Maheswaranathan, B. Cheung, and J. Sohl-Dickstein · 2019
Later among the works it cites.
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
T. Miconi, A. Rawal, J. Clune, and K. O. Stanley · 2019
Later among the works it cites.
Self-supervised learning of pretext-invariant representations
I. Misra and L. van der Maaten · 2019
Later among the works it cites.
Pipedream: Generalized pipeline parallelism for dnn training
D. Narayanan, A. Harlap, A. Phanishayee, V. Seshadri, N. R. Devanur, G. R. Ganger, P. B. Gibbons, and M. Zaharia · 2019
Later among the works it cites.
Measuring the effects of data parallelismon neural network training
C. J. Shallue, J. Lee, J. Antognini, J. Sohl-Dickstein, R. Frostig, and G. E. Dahl · 2019
Later among the works it cites.
Y. Tian, D. Krishnan, and P. Isola · 2019
Later among the works it cites.
Learning to remember from a multi-task teacher
Y. Xiong, M. Ren, and R. Urtasun · 2019
Later among the works it cites.
Local aggregation for unsupervised learning of visual embeddings
C. Zhuang, A. L. Zhai, and D. Yamins · 2019
Later among the works it cites.
A simple framework for contrastive learning of visual representations
T. Chen, S. Kornblith, M. Norouzi, and G. E. Hinton · 2020
Closest in time.
Improved baselines with momentum contrastive learning
X. Chen, H. Fan, R. Girshick, and K. He · 2020
Closest in time.
Unsupervised neural network models of the ventral visual stream
C. Zhuang, S. Yan, A. Nayebi, M. Schrimpf, M. C. Frank, J. J. DiCarlo, and D. L. K. Yamins · 2020
Closest in time.