Deep Learning in Neural Networks: An Overview

Holographic recurrent networks

Plate, T. A. (1993) · 1993

Cited alongside, same era.

Multiprocessor and memory architecture of the neurocomputer SYNAPSE-1

Ramacher, U., Raab, W., Anlauf, J., Hachmann, U., Beichter, J., Bruels, N., Wesseling, M., Sicheneder, E., Maenner, R., Glaess, J., and Wurz, A. (1993) · 1993

Cited alongside, same era.

Redundancy reduction as a strategy for unsupervised learning

Redlich, A. N. (1993) · 1993

Cited alongside, same era.

A direct adaptive method for faster backpropagation learning: The Rprop algorithm

Riedmiller, M. and Braun, H. (1993) · 1993

Cited alongside, same era.

Learning sequential tasks by incrementally adding higher orders

Ring, M. B. (1993) · 1993

Cited alongside, same era.

Continuous history compression

Schmidhuber, J., Mozer, M. C., and Prelinger, D. (1993) · 1993

Cited alongside, same era.

Distinctive image features from scale-invariant key-points

Lowe, D. (2004) · 2004

Later among the works it cites.

GPU implementation of neural networks

Oh, K.-S. and Jung, K. (2004) · 2004

Later among the works it cites.

Reinforcement learning with factored states and actions

Sallans, B. and Hinton, G. (2004) · 2004

Later among the works it cites.

Optimal ordered problem solver

Schmidhuber, J. (2004) · 2004

Later among the works it cites.

A machine learning method for extracting symbolic knowledge from recurrent neural networks

Vahed, A. and Omlin, C. W. (2004) · 2004

Later among the works it cites.

Face localization and tracking in the Neural Abstraction Pyramid

Behnke, S. (2005) · 2005

Later among the works it cites.

Classifying unprompted speech by retraining LSTM nets

Beringer, N., Graves, A., Schiel, F., and Schmidhuber, J. (2005) · 2005

Later among the works it cites.

Parallel and serial neural mechanisms for visual search in macaque area V4

Bichot, N. P., Rossi, A. F., and Desimone, R. (2005) · 2005

Later among the works it cites.

Neurodynamics of biased competition and cooperation for attention: a model with spiking neurons

Deco, G. and Rolls, E. T. (2005) · 2005

Later among the works it cites.

A novel approach for the implementation of large scale spiking neural networks on FPGA hardware

Glackin, B., McGinnity, T. M., Maguire, L. P., Wu, Q., and Belatreche, A. (2005) · 2005

Later among the works it cites.

Reinforcing the driving quality of soccer playing robots by anticipation

Gloye, A., Wiesel, F., Tenchio, O., and Simon, M. (2005) · 2005

Later among the works it cites.

Co-evolving recurrent neurons learn deep memory POMDPs

Gomez, F. J. and Schmidhuber, J. (2005) · 2005

Later among the works it cites.

Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Graves, A. and Schmidhuber, J. (2005) · 2005

Later among the works it cites.

Advances in minimum description length: Theory and applications

Grünwald, P. D., Myung, I. J., and Pitt, M. A. (2005) · 2005

Later among the works it cites.

Sequence classification for protein analysis

Hochreiter, S. and Obermayer, K. (2005) · 2005

Later among the works it cites.

Fast readout of object identity from macaque inferior temporal cortex

Hung, C. P., Kreiman, G., Poggio, T., and DiCarlo, J. J. (2005) · 2005

Later among the works it cites.

Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability

Hutter, M. (2005) · 2005

Later among the works it cites.

Off-road obstacle avoidance through end-to-end learning

LeCun, Y., Muller, U., Cosatto, E., and Flepp, B. (2006) · 2005

Later among the works it cites.

Neural fitted Q iteration—first experiences with a data efficient neural reinforcement learning method

Riedmiller, M. (2005) · 2005

Later among the works it cites.

Intrinsically motivated reinforcement learning

Singh, S., Barto, A. G., and Chentanez, N. (2005) · 2005

Later among the works it cites.

Evolving keepaway soccer players through task decomposition

Whiteson, S., Kohl, N., Miikkulainen, R., and Stone, P. (2005) · 2005

Later among the works it cites.

Loading deep networks is hard: The pyramidal case

Windisch, D. (2005) · 2005

Later among the works it cites.

Pattern Recognition and Machine Learning

Bishop, C. M. (2006) · 2006

Later among the works it cites.

High performance convolutional neural networks for document processing

Chellapilla, K., Puri, S., and Simard, P. (2006) · 2006

Later among the works it cites.

A simple Hebbian/anti-Hebbian network learns the sparse, independent components of natural images

Falconbridge, M. S., Stamps, R. L., and Badcock, D. R. (2006) · 2006

Later among the works it cites.

Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural nets

Graves, A., Fernandez, S., Gomez, F. J., and Schmidhuber, J. (2006) · 2006

Later among the works it cites.

Dimensionality reduction by learning an invariant mapping

Hadsell, R., Chopra, S., and LeCun, Y. (2006) · 2006

Later among the works it cites.

Hierarchical Temporal Memory - Concepts, Theory, and Terminology

Hawkins, J. and George, D. (2006) · 2006

Later among the works it cites.

Reducing the dimensionality of data with neural networks

Hinton, G. and Salakhutdinov, R. (2006) · 2006

Later among the works it cites.

A fast learning algorithm for deep belief nets

Hinton, G. E., Osindero, S., and Teh, Y.-W. (2006) · 2006

Later among the works it cites.

Classification with Bayesian neural networks

Neal, R. M. (2006) · 2006

Later among the works it cites.

High dimensional classification with Bayesian neural networks and Dirichlet diffusion trees

Neal, R. M. and Zhang, J. (2006) · 2006

Later among the works it cites.

Sampling strategies for bag-of-features image classification

Nowak, E., Jurie, F., and Triggs, B. (2006) · 2006

Later among the works it cites.

Efficient learning of sparse representations with an energy-based model

Ranzato, M., Poultney, C., Chopra, S., and LeCun, Y. (2006) · 2006

Later among the works it cites.

Learning long term dependencies with recurrent neural networks

Schäfer, A. M., Udluft, S., and Zimmermann, H.-G. (2006) · 2006

Later among the works it cites.

Implementing synaptic plasticity in a VLSI spiking neural network model

Schemmel, J., Grubl, A., Meier, K., and Mueller, E. (2006) · 2006

Later among the works it cites.

Cross-entropy optimization for independent process analysis

Szabó, Z., Póczos, B., and Lőrincz, A. (2006) · 2006

Later among the works it cites.

Backwards differentiation in AD and neural nets: Past links and new opportunities

Werbos, P. J. (2006) · 2006

Later among the works it cites.

Evolutionary function approximation for reinforcement learning

Whiteson, S. and Stone, P. (2006) · 2006

Later among the works it cites.

A GMDH neural network-based approach to robust fault diagnosis: Application to the DAMADICS benchmark problem

Witczak, M., Korbicz, J., Mrugalski, M., and Patton, R. J. (2006) · 2006

Later among the works it cites.

Greedy layer-wise training of deep networks

Bengio, Y., Lamblin, P., Popovici, D., and Larochelle, H. (2007) · 2007

Later among the works it cites.

Simulation of networks of spiking neurons: a review of tools and strategies

Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J. M., Diesmann, M., Morrison, A., Goodman, P. H., Harris Jr, F. C., et al. (2007) · 2007

Later among the works it cites.

Transformation of shape information in the ventral pathway

Connor, C. E., Brincat, S. L., and Pasupathy, A. (2007) · 2007

Later among the works it cites.

A novel generative encoding for exploiting neural network sensor and output geometry

D’Ambrosio, D. B. and Stanley, K. O. (2007) · 2007

Later among the works it cites.

An application of recurrent neural networks to discriminative keyword spotting

Fernández, S., Graves, A., and Schmidhuber, J. (2007) · 2007

Later among the works it cites.

Sequence labelling in structured domains with hierarchical recurrent neural networks

Fernandez, S., Graves, A., and Schmidhuber, J. (2007) · 2007

Later among the works it cites.

RNN-based Learning of Compact Maps for Efficient Robot Localization

Förster, A., Graves, A., and Schmidhuber, J. (2007) · 2007

Later among the works it cites.

Slowness and sparseness lead to place, head-direction, and spatial-view cells

Franzius, M., Sprekeler, H., and Wiskott, L. (2007) · 2007

Later among the works it cites.

Closed-loop learning of visual control policies

Jodogne, S. R. and Piater, J. H. (2007) · 2007

Later among the works it cites.

Unsupervised learning of invariant feature hierarchies with applications to object recognition

Ranzato, M. A., Huang, F., Boureau, Y., and LeCun, Y. (2007) · 2007

Later among the works it cites.

Prototype resilient, self-modeling robots

Schmidhuber, J. (2007) · 2007

Later among the works it cites.

Training recurrent networks by Evolino

Schmidhuber, J., Wierstra, D., Gagliolo, M., and Gomez, F. J. (2007) · 2007

Later among the works it cites.

An overview of reservoir computing: theory, applications and implementations

Schrauwen, B., Verstraeten, D., and Van Campenhout, J. (2007) · 2007

Later among the works it cites.

Recursive ICA

Shan, H., Zhang, L., and Cottrell, G. W. (2007) · 2007

Later among the works it cites.

Online reservoir adaptation by intrinsic plasticity for backpropagation–decorrelation and echo state learning

Steil, J. J. (2007) · 2007

Later among the works it cites.

A unified architecture for natural language processing: Deep neural networks with multitask learning

Collobert, R. and Weston, J. (2008) · 2008

Later among the works it cites.

Realizing biological spiking network models in a configurable wafer-scale hardware system

Fieres, J., Schemmel, J., and Meier, K. (2008) · 2008

Later among the works it cites.

Accelerated neural evolution through cooperatively coevolved synapses

Gomez, F. J., Schmidhuber, J., and Miikkulainen, R. (2008) · 2008

Later among the works it cites.

Unconstrained on-line handwriting recognition with recurrent neural networks

Graves, A., Fernandez, S., Liwicki, M., Bunke, H., and Schmidhuber, J. (2008) · 2008

Later among the works it cites.

SpiNNaker: mapping neural networks onto a massively-parallel chip multiprocessor

Khan, M. M., Lester, D. R., Plana, L. A., Rast, A., Jin, X., Painkras, E., and Furber, S. B. (2008) · 2008

Later among the works it cites.

Multi-layered GMDH-type neural network self-selecting optimum neural network architecture and its application to 3-dimensional medical image recognition of blood vessels

Kondo, T. and Ueno, J. (2008) · 2008

Later among the works it cites.

Matching categorical object representations in inferior temporal cortex of man and monkey

Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., Tanaka, K., and Bandettini, P. A. (2008) · 2008

Later among the works it cites.

A system for robotic heart surgery that learns to tie knots using recurrent neural networks

Mayer, H., Gomez, F., Wierstra, D., Nagy, I., Knoll, A., and Schmidhuber, J. (2008) · 2008

Later among the works it cites.

State-Dependent Exploration for policy gradient methods

Rückstieß, T., Felder, M., and Schmidhuber, J. (2008) · 2008

Later among the works it cites.

Skill characterization based on betweenness

Simsek, Ö. and Barto, A. G. (2008) · 2008

Later among the works it cites.

The recurrent temporal restricted Boltzmann machine

Sutskever, I., Hinton, G. E., and Taylor, G. W. (2008) · 2008

Later among the works it cites.

A convergent O(n) algorithm for off-policy temporal-difference learning with linear function approximation

Sutton, R. S., Szepesvári, C., and Maei, H. R. (2008) · 2008

Later among the works it cites.

Extracting and composing robust features with denoising autoencoders

Vincent, P., Hugo, L., Bengio, Y., and Manzagol, P.-A. (2008) · 2008

Later among the works it cites.

Unsupervised learning of individuals and categories from images

Waydo, S. and Koch, C. (2008) · 2008

Later among the works it cites.

Natural evolution strategies

Wierstra, D., Schaul, T., Peters, J., and Schmidhuber, J. (2008) · 2008

Later among the works it cites.

Learning to play Go using recursive neural networks

Wu, L. and Baldi, P. (2008) · 2008

Later among the works it cites.

Evolving memory cell structures for sequence learning

Bayer, J., Wierstra, D., Togelius, J., and Schmidhuber, J. (2009) · 2009

Later among the works it cites.

Learning Deep Architectures for AI. Foundations and Trends in Machine Learning, V2(1)

Bengio, Y. (2009) · 2009

Later among the works it cites.

A novel connectionist system for improved unconstrained handwriting recognition

Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., and Schmidhuber, J. (2009) · 2009

Later among the works it cites.

Offline handwriting recognition with multidimensional recurrent neural networks

Graves, A. and Schmidhuber, J. (2009) · 2009

Later among the works it cites.

The Intelligent Movement Machine: An Ethological Perspective on the Primate Motor System

Graziano, M. (2009) · 2009

Later among the works it cites.

The elements of statistical learning

Hastie, T., Tibshirani, R., and Friedman, J. (2009) · 2009

Later among the works it cites.

Neuroevolution strategies for episodic reinforcement learning

Heidrich-Meisner, V. and Igel, C. (2009) · 2009

Later among the works it cites.

Natural image denoising with convolutional networks

Jain, V. and Seung, S. (2009) · 2009

Later among the works it cites.

The 2009 simulated car racing championship

Loiacono, D., Lanzi, P. L., Togelius, J., Onieva, E., Pelta, D. A., Butz, M. V., Lönneker, T. D., Cardamone, L., Perez, D., Sáez, Y., Preuss, M., and Quadflieg, J. (2009) · 2009

Later among the works it cites.

Cartesian genetic programming

Miller, J. F. and Harding, S. L. (2009) · 2009

Later among the works it cites.

Large-scale deep unsupervised learning using graphics processors

Raina, R., Madhavan, A., and Ng, A. (2009) · 2009

Later among the works it cites.

Semantic hashing

Salakhutdinov, R. and Hinton, G. (2009) · 2009

Later among the works it cites.

Caviar: A 45k neuron, 5m synapse, 12g connects/s AER hardware sensory–processing–learning–actuating system for high-speed visual object recognition and tracking

Serrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., Gómez-Rodríguez, F., Camuñas-Mesa, L., Berner, R., Rivas-Pérez, M., Delbruck, T., et al. (2009) · 2009

Later among the works it cites.

A hypercube-based encoding for evolving large-scale neural networks

Stanley, K. O., D’Ambrosio, D. B., and Gauci, J. (2009) · 2009

Later among the works it cites.

Efficient natural evolution strategies

Sun, Y., Wierstra, D., Schaul, T., and Schmidhuber, J. (2009) · 2009

Later among the works it cites.

NNcon: improved protein contact map prediction using 2D-recursive neural networks

Tegge, A. N., Wang, Z., Eickholt, J., and Cheng, J. (2009) · 2009

Later among the works it cites.

Detecting human actions in surveillance videos

Yang, M., Ji, S., Xu, W., Wang, J., Lv, F., Yu, K., Gong, Y., Dikmen, M., Lin, D. J., and Huang, T. S. (2009) · 2009

Later among the works it cites.

Deep machine learning – a new frontier in artificial intelligence research

Arel, I., Rose, D. C., and Karnowski, T. P. (2010) · 2010

Later among the works it cites.

Deep big simple neural nets for handwritten digit recogntion

Ciresan, D. C., Meier, U., Gambardella, L. M., and Schmidhuber, J. (2010) · 2010

Later among the works it cites.

Free-energy based reinforcement learning for vision-based navigation with high-dimensional sensory inputs

Elfwing, S., Otsuka, M., Uchibe, E., and Doya, K. (2010) · 2010

Later among the works it cites.

Why does unsupervised pre-training help deep learning?

Erhan, D., Bengio, Y., Courville, A., Manzagol, P.-A., Vincent, P., and Bengio, S. (2010) · 2010

Later among the works it cites.

Stable adaptive neural network control

Ge, S., Hang, C. C., Lee, T. H., and Zhang, T. (2010) · 2010

Later among the works it cites.

Exponential natural evolution strategies

Glasmachers, T., Schaul, T., Sun, Y., Wierstra, D., and Schmidhuber, J. (2010) · 2010

Later among the works it cites.

Multi-Dimensional Deep Memory Atari-Go Players for Parameter Exploring Policy Gradients

Grüttner, M., Sehnke, F., Schaul, T., and Schmidhuber, J. (2010) · 2010

Later among the works it cites.

Modeling spiking neural networks on SpiNNaker

Jin, X., Lujan, M., Plana, L. A., Davies, S., Temple, S., and Furber, S. B. (2010) · 2010

Later among the works it cites.

Data mining using surface and deep agents based on neural networks

Kak, S., Chen, Y., and Wang, L. (2010) · 2010

Later among the works it cites.

Evolution of neural networks using Cartesian Genetic Programming

Khan, M. M., Khan, G. M., and Miller, J. F. (2010) · 2010

Later among the works it cites.

Evolving neural networks in compressed weight space

Koutník, J., Gomez, F., and Schmidhuber, J. (2010) · 2010

Later among the works it cites.

Deep auto-encoder neural networks in reinforcement learning

Lange, S. and Riedmiller, M. (2010) · 2010

Later among the works it cites.

Reinforcement learning on slow features of high-dimensional input streams

Legenstein, R., Wilbert, N., and Wiskott, L. (2010) · 2010

Later among the works it cites.

Memoir using the chain rule (cited in TMME 7:2&3 p 321-332, 2010)

Leibniz, G. W. (1676) · 2010

Later among the works it cites.

GQ( λ \lambda ): A general gradient algorithm for temporal-difference prediction learning with eligibility traces

Maei, H. R. and Sutton, R. S. (2010) · 2010

Later among the works it cites.

Deep learning via Hessian-free optimization

Martens, J. (2010) · 2010

Later among the works it cites.

Learning to represent spatial transformations with factored higher-order Boltzmann machines

Memisevic, R. and Hinton, G. E. (2010) · 2010

Later among the works it cites.

Phone recognition using restricted Boltzmann machines

Mohamed, A. and Hinton, G. E. (2010) · 2010

Later among the works it cites.

Rectified linear units improve restricted Boltzmann machines

Nair, V. and Hinton, G. E. (2010) · 2010

Later among the works it cites.

Goal-Oriented Representation of the External World: A Free-Energy-Based Approach

Otsuka, M. (2010) · 2010

Later among the works it cites.

Free-energy-based reinforcement learning in a partially observable environment

Otsuka, M., Yoshimoto, J., and Doya, K. (2010) · 2010

Later among the works it cites.

A survey on transfer learning

Pan, S. J. and Yang, Q. (2010) · 2010

Later among the works it cites.

Policy gradient methods

Peters, J. (2010) · 2010

Later among the works it cites.

A convolutional learning system for object classification in 3-D LIDAR data

Prokhorov, D. (2010) · 2010

Later among the works it cites.

Metalearning

Schaul, T. and Schmidhuber, J. (2010) · 2010

Later among the works it cites.

Evaluation of pooling operations in convolutional architectures for object recognition

Scherer, D., Müller, A., and Behnke, S. (2010) · 2010

Later among the works it cites.

Parameter-exploring policy gradients

Sehnke, F., Osendorfer, C., Rückstieß, T., Graves, A., Peters, J., and Schmidhuber, J. (2010) · 2010

Later among the works it cites.

Convolutional networks can learn to generate affinity graphs for image segmentation

Turaga, S. C., Murray, J. F., Jain, V., Roth, F., Helmstaedter, M., Briggman, K., Denk, W., and Seung, H. S. (2010) · 2010

Later among the works it cites.

Recurrent policy gradients

Wierstra, D., Foerster, A., Peters, J., and Schmidhuber, J. (2010) · 2010

Later among the works it cites.

Evolving spiking neural networks for audiovisual information processing

Wysoski, S. G., Benuskova, L., and Kasabov, N. (2010) · 2010

Later among the works it cites.

Autoencoders, unsupervised learning, and deep architectures

Baldi, P. (2012) · 2011

Later among the works it cites.

Learning speaker-specific characteristics with a deep neural architecture

Chen, K. and Salman, A. (2011) · 2011

Later among the works it cites.

On the performance of indirect encoding across the continuum of regularity

Clune, J., Stanley, K. O., Pennock, R. T., and Ofria, C. (2011) · 2011

Later among the works it cites.

Intrinsically motivated evolutionary search for vision-based reinforcement learning

Cuccu, G., Luciw, M., Schmidhuber, J., and Gomez, F. (2011) · 2011

Later among the works it cites.

Adaptive subgradient methods for online learning and stochastic optimization

Duchi, J., Hazan, E., and Singer, Y. (2011) · 2011

Later among the works it cites.

Increasing robustness against background noise: visual pattern recognition by a Neocognitron

Fukushima, K. (2011) · 2011

Later among the works it cites.

Sequential constant size compressor for reinforcement learning

Gisslen, L., Luciw, M., Graziano, V., and Schmidhuber, J. (2011) · 2011

Later among the works it cites.

Deep sparse rectifier networks

Glorot, X., Bordes, A., and Bengio, Y. (2011) · 2011

Later among the works it cites.

Spike-and-slab sparse coding for unsupervised feature discovery

Goodfellow, I. J., Courville, A., and Bengio, Y. (2011) · 2011

Later among the works it cites.

Practical variational inference for neural networks

Graves, A. (2011) · 2011

Later among the works it cites.

Keyword spotting in online handwritten documents containing text and non-text using BLSTM neural networks

Indermuhle, E., Frinken, V., Fischer, A., and Bunke, H. (2011) · 2011

Later among the works it cites.

Neuromorphic silicon neuron circuits

Indiveri, G., Linares-Barranco, B., Hamilton, T. J., Van Schaik, A., Etienne-Cummings, R., Delbruck, T., Liu, S.-C., Dudek, P., Häfliger, P., Renaud, S., et al. (2011) · 2011

Later among the works it cites.

Simulated car racing championship competition software manual

Loiacono, D., Cardamone, L., and Lanzi, P. L. (2011) · 2011

Later among the works it cites.

Learning recurrent neural networks with Hessian-free optimization

Martens, J. and Sutskever, I. (2011) · 2011

Later among the works it cites.

Unsupervised and transfer learning challenge: a deep learning approach

Mesnil, G., Dauphin, Y., Glorot, X., Rifai, S., Bengio, Y., Goodfellow, I., Lavoie, E., Muller, X., Desjardins, G., Warde-Farley, D., Vincent, P., Courville, A., and Bergstra, J. (2011) · 2011

Later among the works it cites.

Sum-product networks: A new deep architecture

Poon, H. and Domingos, P. (2011) · 2011

Later among the works it cites.

Contractive auto-encoders: Explicit invariance during feature extraction

Rifai, S., Vincent, P., Muller, X., Glorot, X., and Bengio, Y. (2011) · 2011

Later among the works it cites.

The two-dimensional organization of behavior

Ring, M., Schaul, T., and Schmidhuber, J. (2011) · 2011

Later among the works it cites.

On fast deep nets for AGI vision

Schmidhuber, J., Ciresan, D., Meier, U., Masci, J., and Graves, A. (2011) · 2011

Later among the works it cites.

Traffic sign recognition with multi-scale convolutional networks

Sermanet, P. and LeCun, Y. (2011) · 2011

Later among the works it cites.

The German traffic sign recognition benchmark: A multi-class classification competition

Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (2011) · 2011

Later among the works it cites.

Learning invariance through imitation

Taylor, G. W., Spiro, I., Bregler, C., and Fergus, R. (2011) · 2011

Later among the works it cites.

On-line driver distraction detection using Long Short-Term Memory

Wöllmer, M., Blaschke, C., Schindl, T., Schuller, B., Färber, B., Mayer, S., and Trefflich, B. (2011) · 2011

Later among the works it cites.

Tikhonov-type regularization for restricted Boltzmann machines

Cho, K., Ilin, A., and Raiko, T. (2012) · 2012

Later among the works it cites.

Multi-column deep neural networks for image classification

Original

Ciresan, D. C., Meier, U., and Schmidhuber, J. (2012c) · 2012

Later among the works it cites.

Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition

Dahl, G., Yu, D., Deng, L., and Acero, A. (2012) · 2012

Later among the works it cites.

Deep architectures for protein contact map prediction

Di Lena, P., Nagata, K., and Baldi, P. (2012) · 2012

Later among the works it cites.

How does the brain solve visual object recognition?

DiCarlo, J. J., Zoccolan, D., and Rust, N. C. (2012) · 2012

Later among the works it cites.

A large-scale model of the functioning brain

Eliasmith, C., Stewart, T. C., Choo, X., Bekolay, T., DeWolf, T., Tang, Y., and Rasmussen, D. (2012) · 2012

Later among the works it cites.

Long-short term memory neural networks language modeling for handwriting recognition

Frinken, V., Zamora-Martinez, F., Espana-Boquera, S., Castro-Bleda, M. J., Fischer, A., and Bunke, H. (2012) · 2012

Later among the works it cites.

Large-scale feature learning with spike-and-slab sparse coding

Goodfellow, I. J., Courville, A. C., and Bengio, Y. (2012) · 2012

Later among the works it cites.

Documenta Mathematica - Extra Volume ISMP

Griewank, A. (2012) · 2012

Later among the works it cites.

A survey of actor-critic reinforcement learning: Standard and natural policy gradients

Grondman, I., Busoniu, L., Lopes, G. A. D., and Babuska, R. (2012) · 2012

Later among the works it cites.

Actor-critic reinforcement learning with energy-based policies

Heess, N., Silver, D., and Teh, Y. W. (2012) · 2012

Later among the works it cites.

IPAL Laboratory and TRIBVN Company and Pitie-Salpetriere Hospital and CIALAB of Ohio State Univ., http://ipal.cnrs.fr/ICPR2012/

ICPR 2012 Contest on Mitosis Detection in Breast Cancer Histological Images (2012) · 2012

Later among the works it cites.

Mode detection in online handwritten documents using BLSTM neural networks

Indermuhle, E., Frinken, V., and Bunke, H. (2012) · 2012

Later among the works it cites.

Incremental slow feature analysis: Adaptive low-complexity slow feature updating from high-dimensional input streams

Kompella, V. R., Luciw, M. D., and Schmidhuber, J. (2012) · 2012

Later among the works it cites.

Imagenet classification with deep convolutional neural networks

Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012) · 2012

Later among the works it cites.

How to Create a Mind: The Secret of Human Thought Revealed

Kurzweil, R. (2012) · 2012

Later among the works it cites.

Building high-level features using large scale unsupervised learning

Le, Q. V., Ranzato, M., Monga, R., Devin, M., Corrado, G., Chen, K., Dean, J., and Ng, A. Y. (2012) · 2012

Later among the works it cites.

The human brain project

Markram, H. (2012) · 2012

Later among the works it cites.

Neural Networks: Tricks of the Trade

Montavon, G., Orr, G., and Müller, K. (2012) · 2012

Later among the works it cites.

Local feature based online mode detection with recurrent neural networks

Otte, S., Krechel, D., Liwicki, M., and Dengel, A. (2012) · 2012

Later among the works it cites.

Deep learning made easier by linear transformations in perceptrons

Raiko, T., Valpola, H., and LeCun, Y. (2012) · 2012

Later among the works it cites.

Autonomous reinforcement learning on raw visual input data in a real world application

Riedmiller, M., Lange, S., and Voigtlaender, A. (2012) · 2012

Later among the works it cites.

A unified approach to evolving plasticity and neural geometry

Risi, S. and Stanley, K. O. (2012) · 2012

Later among the works it cites.

Mitosis detection in breast cancer histological images - an ICPR 2012 contest

Roux, L., Racoceanu, D., Lomenie, N., Kulikova, M., Irshad, H., Klossa, J., Capron, F., Genestie, C., Naour, G. L., and Gurcan, M. N. (2013) · 2012

Later among the works it cites.

Self-delimiting neural networks

Original

Schmidhuber, J. (2012) · 2012

Later among the works it cites.

IEEE International Symposium on Biomedical Imaging (ISBI), http://tinyurl.com/d2fgh7g

Segmentation of Neuronal Structures in EM Stacks Challenge (2012) · 2012

Later among the works it cites.

Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition

Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (2012) · 2012

Later among the works it cites.

Emergence of a ’visual number sense’ in hierarchical generative models

Stoianov, I. and Zorzi, M. (2012) · 2012

Later among the works it cites.

Learning invariance from natural images inspired by observations in the primary visual cortex

Teichmann, M., Wiltschut, J., and Hamker, F. (2012) · 2012

Later among the works it cites.

Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude

Tieleman, T. and Hinton, G. (2012) · 2012

Later among the works it cites.

Reinforcement learning in continuous state and action spaces

van Hasselt, H. (2012) · 2012

Later among the works it cites.

On the computational complexity of stochastic controller optimization in POMDPs

Vlassis, N., Littman, M. L., and Barber, D. (2012) · 2012

Later among the works it cites.

Evolutionary computation for reinforcement learning

Whiteson, S. (2012) · 2012

Later among the works it cites.

Reinforcement Learning

Wiering, M. and van Otterlo, M. (2012) · 2012

Later among the works it cites.

The limits of feedforward vision: Recurrent processing promotes robust object recognition when objects are degraded

Wyatte, D., Curran, T., and O’Reilly, R. (2012) · 2012

Later among the works it cites.

A developmental approach to structural self-organization in reservoir computing

Yin, J., Meng, Y., and Jin, Y. (2012) · 2012

Later among the works it cites.

ADADELTA: An Adaptive Learning Rate Method

Original

Zeiler, M. D. (2012) · 2012

Later among the works it cites.

Forecasting with recurrent neural networks: 12 tricks

Zimmermann, H.-G., Tietz, C., and Grothmann, R. (2012) · 2012

Later among the works it cites.

Adaptive dropout for training deep neural networks

Ba, J. and Frey, B. (2013) · 2013

Later among the works it cites.

On fast dropout and its applicability to recurrent networks

Original

Bayer, J., Osendorfer, C., Chen, N., Urban, S., and van der Smagt, P. (2013) · 2013

Later among the works it cites.

Representation learning: A review and new perspectives

Bengio, Y., Courville, A., and Vincent, P. (2013) · 2013

Later among the works it cites.

Matching recall and storage in sequence learning with spiking neural networks

Brea, J., Senn, W., and Pfister, J.-P. (2013) · 2013

Later among the works it cites.

High-performance OCR for printed English and Fraktur using LSTM networks

Breuel, T. M., Ul-Hasan, A., Al-Azawi, M. A., and Shafait, F. (2013) · 2013

Later among the works it cites.

Enhanced gradient for training restricted Boltzmann machines

Cho, K., Raiko, T., and Ilin, A. (2013) · 2013

Later among the works it cites.

Mitosis detection in breast cancer histology images with deep neural networks

Ciresan, D. C., Giusti, A., Gambardella, L. M., and Schmidhuber, J. (2013) · 2013

Later among the works it cites.

Multi-column deep neural networks for offline handwritten Chinese character classification

Original

Ciresan, D. C. and Schmidhuber, J. (2013) · 2013

Later among the works it cites.

The evolutionary origins of modularity

Clune, J., Mouret, J.-B., and Lipson, H. (2013) · 2013

Later among the works it cites.

Deep learning with COTS HPC systems

Coates, A., Huval, B., Wang, T., Wu, D. J., Ng, A. Y., and Catanzaro, B. (2013) · 2013

Later among the works it cites.

Improving deep neural networks for LVCSR using rectified linear units and dropout

Dahl, G. E., Sainath, T. N., and Hinton, G. E. (2013) · 2013

Later among the works it cites.

DeCAF: A deep convolutional activation feature for generic visual recognition

Original

Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. (2013) · 2013

Later among the works it cites.

How to build a brain: A neural architecture for biological cognition

Eliasmith, C. (2013) · 2013

Later among the works it cites.

How to solve classification and regression problems on high-dimensional data with a supervised extension of slow feature analysis

Escalante-B., A. N. and Wiskott, L. (2013) · 2013

Later among the works it cites.

Real-life voice activity detection with LSTM recurrent neural networks and an application to Hollywood movies

Eyben, F., Weninger, F., Squartini, S., and Schuller, B. (2013) · 2013

Later among the works it cites.

Rich feature hierarchies for accurate object detection and semantic segmentation

Original

Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2013) · 2013

Later among the works it cites.

Fast image scanning with deep max-pooling convolutional neural networks

Giusti, A., Ciresan, D. C., Masci, J., Gambardella, L. M., and Schmidhuber, J. (2013) · 2013

Later among the works it cites.

Maxout networks

Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. (2013) · 2013

Later among the works it cites.

Speech recognition with deep recurrent neural networks

Graves, A., Mohamed, A.-R., and Hinton, G. E. (2013) · 2013

Later among the works it cites.

3D convolutional neural networks for human action recognition

Ji, S., Xu, W., Yang, M., and Yu, K. (2013) · 2013

Later among the works it cites.

Emergence of dynamic memory traces in cortical microcircuit models through STDP

Klampfl, S. and Maass, W. (2013) · 2013

Later among the works it cites.

Evolving large-scale neural networks for vision-based reinforcement learning

Koutník, J., Cuccu, G., Schmidhuber, J., and Gomez, F. (July 2013) · 2013

Later among the works it cites.

Deep hierarchies in the primate visual cortex: What can we learn for computer vision?

Kruger, N., Janssen, P., Kalkan, S., Lappe, M., Leonardis, A., Piater, J., Rodriguez-Sanchez, A., and Wiskott, L. (2013) · 2013

Later among the works it cites.

An intrinsic value system for developing multiple invariant representations with incremental slowness learning

Luciw, M., Kompella, V. R., Kazerounian, S., and Schmidhuber, J. (2013) · 2013

Later among the works it cites.

Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules

Lusci, A., Pollastri, G., and Baldi, P. (2013) · 2013

Later among the works it cites.

Rectifier nonlinearities improve neural network acoustic models

Maas, A. L., Hannun, A. Y., and Ng, A. Y. (2013) · 2013

Later among the works it cites.

A fast learning algorithm for image segmentation with max-pooling convolutional networks

Masci, J., Giusti, A., Ciresan, D. C., Fricout, G., and Schmidhuber, J. (2013) · 2013

Later among the works it cites.

Playing Atari with deep reinforcement learning

Original

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (Dec 2013) · 2013

Later among the works it cites.

Bayesian computation emerges in generic cortical microcircuits through spike-timing-dependent plasticity

Nessler, B., Pfeiffer, M., Buesing, L., and Maass, W. (2013) · 2013

Later among the works it cites.

Real-time classification and sensor fusion with a spiking deep belief network

O’Connor, P., Neil, D., Liu, S.-C., Delbruck, T., and Pfeiffer, M. (2013) · 2013

Later among the works it cites.

Learning and transferring mid-level image representations using convolutional neural networks

Oquab, M., Bottou, L., Laptev, I., and Sivic, J. (2013) · 2013

Later among the works it cites.

Intrinsically motivated learning of real world sensorimotor skills with developmental constraints

Oudeyer, P.-Y., Baranes, A., and Kaplan, F. (2013) · 2013

Later among the works it cites.

Recurrent processing during object recognition

O’Reilly, R. C., Wyatte, D., Herd, S., Mingus, B., and Jilk, D. J. (2013) · 2013

Later among the works it cites.

Regularization and nonlinearities for neural language models: when are they needed?

Original

Pachitariu, M. and Sahani, M. (2013) · 2013

Later among the works it cites.

Dropout Improves Recurrent Neural Networks for Handwriting Recognition

Original

Pham, V., Kermorvant, C., and Louradour, J. (2013) · 2013

Later among the works it cites.

Voxel classification based on triplanar convolutional neural networks applied to cartilage segmentation in knee MRI

Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., and Nielsen, M. (2013) · 2013

Later among the works it cites.

No more pesky learning rates

Schaul, T., Zhang, S., and LeCun, Y. (2013) · 2013

Later among the works it cites.

My first Deep Learning system of 1991 + + Deep Learning timeline 1962-2013

Original

Schmidhuber, J. (2013a) · 2013

Later among the works it cites.

OverFeat: Integrated recognition, localization and detection using convolutional networks

Original

Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., and LeCun, Y. (2013) · 2013

Later among the works it cites.

Compete to compute

Srivastava, R. K., Masci, J., Kazerounian, S., Gomez, F., and Schmidhuber, J. (2013) · 2013

Later among the works it cites.

A Linear Time Natural Evolution Strategy for Non-Separable Functions

Sun, Y., Gomez, F., Schaul, T., and Schmidhuber, J. (2013) · 2013

Later among the works it cites.

Deep neural networks for object detection

Szegedy, C., Toshev, A., and Erhan, D. (2013) · 2013

Later among the works it cites.

Cartesian Genetic Programming encoded artificial neural networks: A comparison using three benchmarks

Turner, A. J. and Miller, J. F. (2013) · 2013

Later among the works it cites.

Critical factors in the performance of HyperNEAT

van den Berg, T. and Whiteson, S. (2013) · 2013

Later among the works it cites.

MICCAI 2013 Grand Challenge on Mitosis Detection

Veta, M., Viergever, M., Pluim, J., Stathonikos, N., and van Diest, P. J. (2013) · 2013

Later among the works it cites.

Fast dropout training

Wang, S. and Manning, C. (2013) · 2013

Later among the works it cites.

Keyword spotting exploiting Long Short-Term Memory

Wöllmer, M., Schuller, B., and Rigoll, G. (2013) · 2013

Later among the works it cites.

Hierarchical modular optimization of convolutional networks achieves representations similar to macaque IT and human ventral stream

Yamins, D., Hong, H., Cadieu, C., and DiCarlo, J. J. (2013) · 2013

Later among the works it cites.

ICDAR 2013 Chinese handwriting recognition competition

Yin, F., Wang, Q.-F., Zhang, X.-Y., and Liu, C.-L. (2013) · 2013

Later among the works it cites.

Visualizing and understanding convolutional networks

Original

Zeiler, M. D. and Fergus, R. (2013) · 2013

Later among the works it cites.

The dropout learning algorithm

Baldi, P. and Sadowski, P. (2014) · 2014

Closest in time.

Variational inference of latent state sequences using recurrent networks

Original

Bayer, J. and Osendorfer, C. (2014) · 2014

Closest in time.

The A2iA Arabic Handwritten Text Recognition System at the OpenHaRT2013 Evaluation

Bluche, T., Louradour, J., Knibbe, M., Moysset, B., Benzeghiba, F., and Kermorvant, C. (2014) · 2014

Closest in time.

Social signal classification using deep BLSTM recurrent neural networks

Brueckner, R. and Schulter, B. (2014) · 2014

Closest in time.

Foundations and Advances in Deep Learning

Cho, K. (2014) · 2014

Closest in time.

Deep Learning: Methods and Applications

Deng, L. and Yu, D. (2014) · 2014

Closest in time.

TTS synthesis with bidirectional LSTM based recurrent neural networks

Fan, Y., Qian, Y., Xie, F., and Soong, F. K. (2014) · 2014

Closest in time.

Prosody contour prediction with Long Short-Term Memory, bi-directional, deep recurrent neural networks

Fernandez, R., Rendel, A., Ramabhadran, B., and Hoory, R. (2014) · 2014

Closest in time.

Training restricted Boltzmann machines: An introduction

Fischer, A. and Igel, C. (2014) · 2014

Closest in time.

Robust speech recognition using long short-term memory recurrent neural networks for hybrid acoustic modelling

Geiger, J. T., Zhang, Z., Weninger, F., Schuller, B., and Rigoll, G. (2014) · 2014

Closest in time.

Automatic language identification using Long Short-Term Memory recurrent neural networks

Gonzalez-Dominguez, J., Lopez-Moreno, I., Sak, H., Gonzalez-Rodriguez, J., and Moreno, P. J. (2014) · 2014

Closest in time.

Towards end-to-end speech recognition with recurrent neural networks

Graves, A. and Jaitly, N. (2014) · 2014

Closest in time.

Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning

Guo, X., Singh, S., Lee, H., Lewis, R., and Wang, X. (2014) · 2014

Closest in time.

Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning

Hoerzer, G. M., Legenstein, R., and Maass, W. (2014) · 2014

Closest in time.

Large-scale video classification with convolutional neural networks

Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei, L. (2014) · 2014

Closest in time.

Neucube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data

Kasabov, N. K. (2014) · 2014

Closest in time.

Automatic feature learning for robust shadow detection

Khan, S. H., Bennamoun, M., Sohel, F., and Togneri, R. (2014) · 2014

Closest in time.

A Clockwork RNN

Original

Koutník, J., Greff, K., Gomez, F., and Schmidhuber, J. (2014) · 2014

Closest in time.

Deep learning based imaging data completion for improved brain disease diagnosis

Li, R., Zhang, W., Suk, H.-I., Wang, L., Li, J., Shen, D., and Ji, S. (2014) · 2014

Closest in time.

Multi-resolution linear prediction based features for audio onset detection with bidirectional LSTM neural networks

Marchi, E., Ferroni, G., Eyben, F., Gabrielli, L., Squartini, S., and Schuller, B. (2014) · 2014

Closest in time.

A million spiking-neuron integrated circuit with a scalable communication network and interface

Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., Jackson, B. L., Imam, N., Guo, C., Nakamura, Y., Brezzo, B., Vo, I., Esser, S. K., Appuswamy, R., Taba, B., Amir, A., Flickner, M. D., Risk, W. P., Manohar, R., and Modha, D. S. (2014) · 2014

Closest in time.

Event-driven contrastive divergence for spiking neuromorphic systems

Neftci, E., Das, S., Pedroni, B., Kreutz-Delgado, K., and Cauwenberghs, G. (2014) · 2014

Closest in time.

Minitaur, an event-driven FPGA-based spiking network accelerator

Neil, D. and Liu, S.-C. (2014) · 2014

Closest in time.

CNN features off-the-shelf: an astounding baseline for recognition

Original

Razavian, A. S., Azizpour, H., Sullivan, J., and Carlsson, S. (2014) · 2014

Closest in time.

Stochastic variational learning in recurrent spiking networks

Rezende, D. J. and Gerstner, W. (2014) · 2014

Closest in time.

Efficient visual coding: From retina to V2

Original

Shan, H. and Cottrell, G. (2014) · 2014

Closest in time.

Learning deep and wide: A spectral method for learning deep networks

Shao, L., Wu, D., and Li, X. (2014) · 2014

Closest in time.

Sequence to sequence learning with neural networks

Original

Sutskever, I., Vinyals, O., and Le, Q. V. (2014) · 2014

Closest in time.

Going deeper with convolutions

Original

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2014) · 2014

Closest in time.

Leveraging hierarchical parametric networks for skeletal joints based action segmentation and recognition

Wu, D. and Shao, L. (2014) · 2014

Closest in time.

Hierarchical spatiotemporal feature extraction using recurrent online clustering

Young, S., Davis, A., Mishtal, A., and Arel, I. (2014) · 2014

Closest in time.

Neural network language models for off-line handwriting recognition

Zamora-Martínez, F., Frinken, V., España-Boquera, S., Castro-Bleda, M., Fischer, A., and Bunke, H. (2014) · 2014

Closest in time.

Adaptive behavior with fixed weights in RNN: an overview

Prokhorov, D. V., Feldkamp, L. A., and Tyukin, I. Y. (2002) · 2023

Closest in time.

Coding of color and form in the geniculostriate visual pathway

Lennie, P. and Movshon, J. A. (2005) · 2033

Closest in time.

Stimulus-selective properties of inferior temporal neurons in the macaque

Desimone, R., Albright, T. D., Gross, C. G., and Bruce, C. (1984) · 2062

Closest in time.

An active pulse transmission line simulating nerve axon

Nagumo, J., Arimoto, S., and Yoshizawa, S. (1962) · 2070

Closest in time.