Fetching the paper…
Reading the bibliography…
We explore whether useful temporal neural generative models can be learned from sequential data without back-propagation through time.
On information and sufficiency
Kullback, S. and Leibler, R. A. (1951) · 1951
Earlier work this paper cites.
How does a brain build a cognitive code?
Grossberg, S. (1982) · 1982
Earlier work this paper cites.
Neural networks and physical systems with emergent collective computational abilities
Hopfield, J. J. (1982) · 1982
Earlier work this paper cites.
Competitive learning: From interactive activation to adaptive resonance
Grossberg, S. (1987) · 1987
Earlier work this paper cites.
Learning representations by recirculation
Hinton, G. E. and McClelland, J. L. (1988) · 1988
Earlier work this paper cites.
Unsupervised learning
Barlow, H. B. (1989) · 1989
Earlier work this paper cites.
Generalization and network design strategies
LeCun, Y. et al. (1989) · 1989
Earlier work this paper cites.
Implicit learning and tacit knowledge
Reber, A. S. (1989) · 1989
Earlier work this paper cites.
A learning algorithm for continually running fully recurrent neural networks
Williams, R. J. and Zipser, D. (1989) · 1989
Earlier work this paper cites.
Acceleration of stochastic approximation by averaging
Polyak, B. T. and Juditsky, A. B. (1992) · 1992
Earlier work this paper cites.
The" wake-sleep" algorithm for unsupervised neural networks
Hinton, G. E., Dayan, P., Frey, B. J., and Neal, R. M. (1995) · 1995
Earlier work this paper cites.
Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm
O’Reilly, R. C. (1996) · 1996
Earlier work this paper cites.
Sparse coding with an overcomplete basis set: A strategy employed by v1?
Olshausen, B. A. and Field, D. J. (1997) · 1997
Earlier work this paper cites.
Dynamic model of visual recognition predicts neural response properties in the visual cortex
Rao, R. P. and Ballard, D. H. (1997) · 1997
Earlier work this paper cites.
Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects
Rao, R. P. and Ballard, D. H. (1999) · 1999
Earlier work this paper cites.
Can sequence learning be implicit? new evidence with the process dissociation procedure
Destrebecqz, A. and Cleeremans, A. (2001) · 2001
Earlier work this paper cites.
Spiking neuron models: Single neurons, populations, plasticity
Andrew, A. M. (2003) · 2003
Earlier work this paper cites.
Equivalence of backpropagation and contrastive hebbian learning in a layered network
Xie, X. and Seung, H. S. (2003) · 2003
Earlier work this paper cites.
Training products of experts by minimizing contrastive divergence
Hinton, G. E. (2006) · 2006
Earlier work this paper cites.
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H., et al. (2007) · 2007
Earlier work this paper cites.
Modeling human motion using binary latent variables
Taylor, G. W., Hinton, G. E., and Roweis, S. T. (2007) · 2007
Earlier work this paper cites.
The recurrent temporal restricted boltzmann machine
Sutskever, I., Hinton, G. E., and Taylor, G. W. (2009) · 2009
Earlier work this paper cites.
Fast inference in sparse coding algorithms with applications to object recognition
Kavukcuoglu, K., Ranzato, M., and LeCun, Y. (2010) · 2010
Earlier work this paper cites.
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Cernockỳ, J., and Khudanpur, S. (2010) · 2010
Cited alongside, same era.
Efficient learning of deep boltzmann machines
Salakhutdinov, R. and Larochelle, H. (2010) · 2010
Cited alongside, same era.
Deep sparse rectifier neural networks
Glorot, X., Bordes, A., and Bengio, Y. (2011) · 2011
Cited alongside, same era.
Predictive coding
Huang, Y. and Rao, R. P. (2011) · 2011
Cited alongside, same era.
Structured sparse coding via lateral inhibition
Szlam, A. D., Gregor, K., and LeCun, Y. L. (2011) · 2011
Cited alongside, same era.
Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. (2012) · 2012
Cited alongside, same era.
Deep temporal sigmoid belief networks for sequence modeling
Gan, Z., Li, C., Henao, R., Carlson, D. E., and Carin, L. (2015) · 2015
Later among the works it cites.
Deep learning
LeCun, Y., Bengio, Y., and Hinton, G. (2015) · 2015
Later among the works it cites.
Difference target propagation
Lee, D.-H., Zhang, S., Fischer, A., and Bengio, Y. (2015) · 2015
Later among the works it cites.
Never ending learning
Mitchell, T. M., Cohen, W. W., Hruschka Jr, E. R., Talukdar, P. P., Betteridge, J., Carlson, A., Mishra, B. D., Gardner, M., Kisiel, B., Krishnamurthy, J., et al. (2015) · 2015
Later among the works it cites.
Training recurrent networks online without backtracking
Ollivier, Y., Tallec, C., and Charpiat, G. (2015) · 2015
Later among the works it cites.
Learning a deep hybrid model for semi-supervised text classification
Ororbia II, A. G., Giles, C. L., and Reitter, D. (2015a) · 2015
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Spike-and-slab sparse coding for unsupervised feature discovery
Goodfellow, I. J., Courville, A., and Bengio, Y. (2012) · 2012
Cited alongside, same era.
Deep learning of representations: Looking forward
Bengio, Y. (2013) · 2013
Cited alongside, same era.
Deep predictive coding networks
Chalasani, R. and Principe, J. C. (2013) · 2013
Cited alongside, same era.
Simple sparsification improves sparse denoising autoencoders in denoising highly corrupted images
Cho, K. (2013) · 2013
Cited alongside, same era.
Whatever next? predictive brains, situated agents, and the future of cognitive science
Clark, A. (2013) · 2013
Cited alongside, same era.
Auto-encoding variational bayes
Kingma, D. P. and Welling, M. (2013) · 2013
Cited alongside, same era.
Later among the works it cites.
Online learning of deep hybrid architectures for semi-supervised categorization
Ororbia II, A. G., Reitter, D., Wu, J., and Giles, C. L. (2015c) · 2015
Later among the works it cites.
van Rooyen, B. and Williamson, R. C. (2015) · 2015
Later among the works it cites.
Learning in the machine: Random backpropagation and the learning channel
Baldi, P., Sadowski, P., and Lu, Z. (2016) · 2016
Later among the works it cites.
The variational walkback algorithm
Goyal, A., Ke, N. R., Lamb, A., and Bengio, Y. (2016) · 2016
Later among the works it cites.
Decoupled neural interfaces using synthetic gradients
Jaderberg, M., Czarnecki, W. M., Osindero, S., Vinyals, O., Graves, A., and Kavukcuoglu, K. (2016) · 2016
Later among the works it cites.
How important is weight symmetry in backpropagation?
Liao, Q., Leibo, J. Z., and Poggio, T. A. (2016) · 2016
Later among the works it cites.
Deep predictive coding networks for video prediction and unsupervised learning
Lotter, W., Kreiman, G., and Cox, D. (2016) · 2016
Later among the works it cites.
A modular theory of feature learning
McNamara, D., Ong, C. S., and Williamson, R. C. (2016) · 2016
Later among the works it cites.
Direct feedback alignment provides learning in deep neural networks
Nøkland, A. (2016) · 2016
Later among the works it cites.
What Auto-encoders Could Learn from Brains
van den Broeke, G. (2016) · 2016
Later among the works it cites.
Event-driven random back-propagation: Enabling neuromorphic deep learning machines
Neftci, E. O., Augustine, C., Paul, S., and Detorakis, G. (2017) · 2017
Closest in time.
Exploiting spatio-temporal structure with recurrent winner-take-all networks
Santana, E., Emigh, M. S., Zegers, P., and Principe, J. C. (2017) · 2017
Closest in time.
Equilibrium propagation: Bridging the gap between energy-based models and backpropagation
Scellier, B. and Bengio, Y. (2017) · 2017
Closest in time.
Piecewise latent variables for neural variational text processing
Serban, I. V., Ororbia II, A. G., Pineau, J., and Courville, A. (2017) · 2017
Closest in time.
Unbiased online recurrent optimization
Tallec, C. and Ollivier, Y. (2017) · 2017
Closest in time.
Training language models using target-propagation
Wiseman, S., Chopra, S., Ranzato, M., Szlam, A., Sun, R., Chintala, S., and Vasilache, N. (2017) · 2017
Closest in time.