2019

AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence

Clune, Jeff

Understand

Perhaps the most ambitious scientific quest in human history is the creation of general artificial intelligence, which roughly means AI that is as smart or smarter than humans.

The dominant approach in the machine learning community is to attempt to discover each of the pieces required for intelligence, with the implicit assumption that some future group will complete the Herculean task of figuring out how to combine all of those pieces into a complex thinking machine.
I call this the "manual AI approach".
This paper describes another exciting path that ultimately may be more successful at producing general AI.

Built on

Bootstrapping on the adaptive landscape
Michael Conrad · 1979
Earlier work this paper cites.
Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-… hook
Jürgen Schmidhuber · 1987
Earlier work this paper cites.
Alvinn: An autonomous land vehicle in a neural network
Dean A Pomerleau · 1989
Earlier work this paper cites.
Fixed-weight networks can learn
Neil E Cotter and Peter R Conwell · 1990
Earlier work this paper cites.
Designing application-specific neural networks using the genetic algorithm
Steven A Harp, Tariq Samad, and Aloke Guha · 1990
Earlier work this paper cites.
Optimal brain damage
Yann LeCun, John S Denker, and Sara A Solla · 1990
Earlier work this paper cites.
Curious model-building control systems
Jürgen Schmidhuber · 1991
Earlier work this paper cites.
Semi-distributed representations and catastrophic forgetting in connectionist networks
Robert M French · 1992
Earlier work this paper cites.
Byte-sized evolution
John Maynard Smith · 1992
Earlier work this paper cites.
Feudal reinforcement learning
Peter Dayan and Geoffrey E Hinton · 1993
Earlier work this paper cites.
An evolutionary approach to synthetic biology: Zen and the art of creating life
Thomas S Ray · 1993
Earlier work this paper cites.
Automatic definition of modular neural networks
F. Gruau · 1994
Earlier work this paper cites.
Evolving 3D morphology and behavior by competition
K. Sims · 1994
Earlier work this paper cites.
Artificial intelligence: a modern approach , volume 74
S. J. Russell, P. Norvig, J. F Canny, J. M. Malik, and D. D. Edwards · 1995
Earlier work this paper cites.
On the computational power of neural nets
Hava T Siegelmann and Eduardo D Sontag · 1995
Earlier work this paper cites.
Perspective: complex adaptations and the evolution of evolvability
Günter P Wagner and Lee Altenberg · 1996
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Artificial life: An overview
Christopher G Langton · 1997
Earlier work this paper cites.
Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states
S.G. Ficici and J.B. Pollack · 1998
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Reinforcement learning: An introduction
Richard S Sutton, Andrew G Barto, et al · 1998
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
Robert M French · 1999
Earlier work this paper cites.
Object recognition from local scale-invariant features
David G Lowe et al · 1999
Earlier work this paper cites.
Evolving artificial neural networks
Xin Yao · 1999
Earlier work this paper cites.
Evolution of biological complexity
C. Adami, C. Ofria, and T.C. Collier · 2000
Earlier work this paper cites.
Open problems in artificial life
Mark A Bedau, John S McCaskill, Norman H Packard, Steen Rasmussen, Chris Adami, David G Green, Takashi Ikegami, Kunihiko Kaneko, and Thomas S Ray · 2000
Earlier work this paper cites.
Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit
Richard HR Hahnloser, Rahul Sarpeshkar, Misha A Mahowald, Rodney J Douglas, and H Sebastian Seung · 2000
Earlier work this paper cites.
A theory of universal artificial intelligence based on algorithmic complexity
Marcus Hutter · 2000
Earlier work this paper cites.
Algorithms for inverse reinforcement learning
Andrew Y Ng and Stuart J Russell · 2000
Earlier work this paper cites.
Learning to learn using gradient descent
Sepp Hochreiter, A Steven Younger, and Peter R Conwell · 2001
Earlier work this paper cites.
An empirical analysis of collaboration methods in cooperative coevolutionary algorithms
R. Paul Wiegand, William C. Liles, and Kenneth A. De Jong · 2001
Earlier work this paper cites.
Creating high-level components with a generative representation for body-brain evolution
G.S. Hornby and J.B. Pollack · 2002
Earlier work this paper cites.
Evolving neural networks through augmenting topologies
K.O. Stanley and R. Miikkulainen · 2002
Earlier work this paper cites.
Recent advances in hierarchical reinforcement learning
Andrew G Barto and Sridhar Mahadevan · 2003
Earlier work this paper cites.
The evolutionary origin of complex features
R. E. Lenski, C. Ofria, R. T. Pennock, and C. Adami · 2003
Earlier work this paper cites.
A taxonomy for artificial embryogeny
K.O. Stanley and R. Miikkulainen · 2003
Earlier work this paper cites.
Apprenticeship learning via inverse reinforcement learning
Pieter Abbeel and Andrew Y Ng · 2004
Earlier work this paper cites.
The incremental pareto-coevolution archive
Edwin D De Jong · 2004
Earlier work this paper cites.
Universal artificial intelligence: Sequential decisions based on algorithmic probability
Marcus Hutter · 2004
Earlier work this paper cites.
Ideal evaluation from coevolution
Edwin D De Jong and Jordan B Pollack · 2004
Earlier work this paper cites.
Histograms of oriented gradients for human detection
Navneet Dalal and Bill Triggs · 2005
Earlier work this paper cites.
Principles of brain evolution
G.F. Striedter · 2005
Earlier work this paper cites.
Intrinsic motivation systems for autonomous mental development
Pierre-Yves Oudeyer, Frdric Kaplan, and Verena V Hafner · 2007
Earlier work this paper cites.
Compositional pattern producing networks: A novel abstraction of development
K.O. Stanley · 2007
Earlier work this paper cites.
Is evolvability evolvable?
M. Pigliucci · 2008
Earlier work this paper cites.
Overcoming the bootstrap problem in evolutionary robotics using behavioral diversity
Jean-Baptiste Mouret and Stéphane Doncieux · 2009
Earlier work this paper cites.
What is intrinsic motivation? a typology of computational approaches
Pierre-Yves Oudeyer and Frederic Kaplan · 2009
Earlier work this paper cites.
How novelty search escapes the deceptive trap of learning to learn
Sebastian Risi, Sandy D Vanderbleek, Charles E Hughes, and Kenneth O Stanley · 2009
Earlier work this paper cites.
A hypercube-based encoding for evolving large-scale neural networks
K.O. Stanley, D.B. D’Ambrosio, and J. Gauci · 2009
Earlier work this paper cites.
The cultural origins of human cognition
Michael Tomasello · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio · 2010
Earlier work this paper cites.
Efficiently evolving programs through the search for novelty
Joel Lehman and Kenneth O Stanley · 2010
Earlier work this paper cites.
Rectified linear units improve restricted boltzmann machines
Vinod Nair and Geoffrey E Hinton · 2010
Earlier work this paper cites.
Formal theory of creativity, fun, and intrinsic motivation (1990–2010)
Jürgen Schmidhuber · 2010
Earlier work this paper cites.
Computer Vision: Algorithms and Applications
Richard Szeliski · 2010
Earlier work this paper cites.
Transforming auto-encoders
Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang · 2011
Earlier work this paper cites.
Building high-level features using large scale unsupervised learning
Original
Quoc V Le, Marc’Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen, Greg S Corrado, Jeff Dean, and Andrew Y Ng · 2011
Earlier work this paper cites.
Improving evolvability through novelty search and self-adaptation
Joel Lehman and Kenneth O Stanley · 2011
Earlier work this paper cites.
On the deleterious effects of a priori objectives on evolution and representation
B.G. Woolley and K.O. Stanley · 2011
Earlier work this paper cites.
On the relationship between environmental and morphological complexity in evolved robots
Joshua E Auerbach and Joshua C Bongard · 2012
Earlier work this paper cites.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al · 2012
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. Hinton · 2012
Earlier work this paper cites.
Encouraging behavioral diversity in evolutionary robotics: an empirical study
J.-B. Mouret and S. Doncieux · 2012
Earlier work this paper cites.
Coevolutionary Principles , pages 987–1033
Elena Popovici, Anthony Bucci, R. Paul Wiegand, and Edwin D. De Jong · 2012
Earlier work this paper cites.

Similar

Practical bayesian optimization of machine learning algorithms
Jasper Snoek, Hugo Larochelle, and Ryan P Adams · 2012
Cited alongside, same era.
From modulated hebbian plasticity to simple behavior learning through noise and weight saturation
Andrea Soltoggio and Kenneth O Stanley · 2012
Cited alongside, same era.
Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding
N. Cheney, R. MacCurdy, J. Clune, and H. Lipson · 2013
Cited alongside, same era.
The evolutionary origins of modularity
J. Clune, J-B. Mouret, and H. Lipson · 2013
Cited alongside, same era.
Rectifier nonlinearities improve neural network acoustic models
Andrew L Maas, Awni Y Hannun, and Andrew Y Ng · 2013
Cited alongside, same era.
The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects
Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin · 2013
Cited alongside, same era.
Powerplay: Training an increasingly general problem solver by continually searching for the simplest still unsolvable problem
Jürgen Schmidhuber · 2013
Cited alongside, same era.
First experiments with powerplay
Rupesh Kumar Srivastava, Bas R Steunebrink, and Jürgen Schmidhuber · 2013
Cited alongside, same era.
Intriguing properties of neural networks
Original
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus · 2013
Cited alongside, same era.
Robustness and evolvability in living systems , volume 24
Andreas Wagner · 2013
Cited alongside, same era.
Neural machine translation by jointly learning to align and translate
Original
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio · 2014
Cited alongside, same era.
The ethics of artificial intelligence
Nick Bostrom and Eliezer Yudkowsky · 2014
Cited alongside, same era.

Then

Neural architecture search with reinforcement learning
Original
Barret Zoph and Quoc V Le · 2016
Later among the works it cites.
Accelerating neural architecture search using performance prediction
Original
Bowen Baker, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik · 2017
Later among the works it cites.
Emergent complexity via multi-agent competition
Original
Trapit Bansal, Jakub Pachocki, Szymon Sidor, Ilya Sutskever, and Igor Mordatch · 2017
Later among the works it cites.
Building machines that learn and think for themselves: Commentary on lake et al., behavioral and brain sciences, 2017
Original
Matthew Botvinick, David GT Barrett, Peter Battaglia, Nando de Freitas, Dharshan Kumaran, Joel Z Leibo, Tim Lillicrap, Joseph Modayil, S Mohamed, Neil C Rabinowitz, et al · 2017
Later among the works it cites.
Minimal criterion coevolution: a new approach to open-ended search
Jonathan C Brant and Kenneth O Stanley · 2017
Later among the works it cites.
Meta-learning and universality: Deep representations and gradient descent can approximate any learning algorithm
Original
Chelsea Finn and Sergey Levine · 2017
Later among the works it cites.
Model-agnostic meta-learning for fast adaptation of deep networks
Chelsea Finn, Pieter Abbeel, and Sergey Levine · 2017
Later among the works it cites.
Noisy networks for exploration
Original
Meire Fortunato, Mohammad Gheshlaghi Azar, Bilal Piot, Jacob Menick, Ian Osband, Alex Graves, Vlad Mnih, Remi Munos, Demis Hassabis, Olivier Pietquin, et al · 2017
Later among the works it cites.
Visualizing and understanding atari agents
Original
Sam Greydanus, Anurag Koul, Jonathan Dodge, and Alan Fern · 2017
Later among the works it cites.
Rainbow: Combining improvements in deep reinforcement learning
Original
Matteo Hessel, Joseph Modayil, Hado Van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, and David Silver · 2017
Later among the works it cites.
Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger · 2017
Later among the works it cites.
Population based training of neural networks
Original
Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, et al · 2017
Later among the works it cites.
Overcoming catastrophic forgetting in neural networks
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al · 2017
Later among the works it cites.
Learning curve prediction with bayesian neural networks
Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, and Frank Hutter · 2017
Later among the works it cites.
How evolution learns to generalise: Using the principles of learning theory to understand the evolution of developmental organisation
Kostas Kouvaris, Jeff Clune, Louis Kounios, Markus Brede, and Richard A Watson · 2017
Later among the works it cites.
Evolving deep neural networks
Original
Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Dan Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, and Babak Hodjat · 2017
Later among the works it cites.
Plug & play generative networks: Conditional iterative generation of images in latent space
Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski · 2017
Later among the works it cites.
Curiosity-driven exploration by self-supervised prediction
Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell · 2017
Later among the works it cites.
Parameter space noise for exploration
Original
Matthias Plappert, Rein Houthooft, Prafulla Dhariwal, Szymon Sidor, Richard Y Chen, Xi Chen, Tamim Asfour, Pieter Abbeel, and Marcin Andrychowicz · 2017
Later among the works it cites.
Optimization as a model for few-shot learning
Sachin Ravi and Hugo Larochelle · 2017
Later among the works it cites.
Dynamic routing between capsules
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton · 2017
Later among the works it cites.
Evolution strategies as a scalable alternative to reinforcement learning
Original
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever · 2017
Later among the works it cites.
Mastering chess and shogi by self-play with a general reinforcement learning algorithm
Original
David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al · 2017
Later among the works it cites.
Third-person imitation learning
Original
Bradly C Stadie, Pieter Abbeel, and Ilya Sutskever · 2017
Later among the works it cites.
Open-endedness: The last grand challenge you’ve never heard of
KO Stanley, J Lehman, and L Soros · 2017
Later among the works it cites.
Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning
Original
Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Kenneth O Stanley, and Jeff Clune · 2017
Later among the works it cites.
Inception-v4, inception-resnet and the impact of residual connections on learning
Christian Szegedy, Sergey Ioffe, and Vincent Vanhoucke · 2017
Later among the works it cites.
Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks
Roby Velez and Jeff Clune · 2017
Later among the works it cites.
Feudal networks for hierarchical reinforcement learning
Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, and Koray Kavukcuoglu · 2017
Later among the works it cites.
Continual learning through synaptic intelligence
Friedemann Zenke, Ben Poole, and Surya Ganguli · 2017
Later among the works it cites.
Playing hard exploration games by watching youtube
Yusuf Aytar, Tobias Pfaff, David Budden, Thomas Paine, Ziyu Wang, and Nando de Freitas · 2018
Later among the works it cites.
Learning from demonstration in the wild
Original
Feryal Behbahani, Kyriacos Shiarlis, Xi Chen, Vitaly Kurin, Sudhanshu Kasewa, Ciprian Stirbu, João Gomes, Supratik Paul, Frans A Oliehoek, João Messias, et al · 2018
Later among the works it cites.
Large-scale study of curiosity-driven learning
Original
Yuri Burda, Harri Edwards, Deepak Pathak, Amos Storkey, Trevor Darrell, and Alexei A Efros · 2018
Later among the works it cites.
Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty-seeking agents
Edoardo Conti, Vashisht Madhavan, Felipe Petroski Such, Joel Lehman, Kenneth Stanley, and Jeff Clune · 2018
Later among the works it cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Later among the works it cites.
Agi safety literature review
Original
Tom Everitt, Gary Lea, and Marcus Hutter · 2018
Later among the works it cites.
Diversity is all you need: Learning skills without a reward function
Original
Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine · 2018
Later among the works it cites.
Unsupervised meta-learning for reinforcement learning
Original
Abhishek Gupta, Benjamin Eysenbach, Chelsea Finn, and Sergey Levine · 2018
Later among the works it cites.
Reinforcement learning for improving agent design
Original
David Ha · 2018
Later among the works it cites.
Learning an embedding space for transferable robot skills
Karol Hausman, Jost Tobias Springenberg, Ziyu Wang, Nicolas Heess, and Martin Riedmiller · 2018
Later among the works it cites.
Evolving multimodal robot behavior via many stepping stones with the combinatorial multi-objective evolutionary algorithm
Original
Joost Huizinga and Jeff Clune · 2018
Later among the works it cites.
The emergence of canalization and evolvability in an open-ended, interactive evolutionary system
Joost Huizinga, Kenneth O Stanley, and Jeff Clune · 2018
Later among the works it cites.
Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
Original
Max Jaderberg, Wojciech M Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castaneda, Charles Beattie, Neil C Rabinowitz, Ari S Morcos, Avraham Ruderman, et al · 2018
Later among the works it cites.
Illuminating generalization in deep reinforcement learning through procedural level generation
Original
Niels Justesen, Ruben Rodriguez Torrado, Philip Bontrager, Ahmed Khalifa, Julian Togelius, and Sebastian Risi · 2018
Later among the works it cites.
Neural architecture search with bayesian optimisation and optimal transport
Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, and Eric P Xing · 2018
Later among the works it cites.
Progressive neural architecture search
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy · 2018
Later among the works it cites.
Differentiable plasticity: training plastic neural networks with backpropagation
Thomas Miconi, Jeff Clune, and Kenneth O Stanley · 2018
Later among the works it cites.
From nodes to networks: Evolving recurrent neural networks
Original
Aditya Rawal and Risto Miikkulainen · 2018
Later among the works it cites.
Learning montezuma’s revenge from a single demonstration
Original
Tim Salimans and Richard Chen · 2018
Later among the works it cites.
Episodic curiosity through reachability
Original
Nikolay Savinov, Anton Raichuk, Raphaël Marinier, Damien Vincent, Marc Pollefeys, Timothy Lillicrap, and Sylvain Gelly · 2018
Later among the works it cites.
Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks
Andrea Soltoggio, Kenneth O Stanley, and Sebastian Risi · 2018
Later among the works it cites.
Deep curiosity search: Intra-life exploration improves performance on challenging deep reinforcement problems
Christopher Stanton and Jeff Clune · 2018
Later among the works it cites.
Behavioral cloning from observation
Original
Faraz Torabi, Garrett Warnell, and Peter Stone · 2018
Later among the works it cites.
Maximum individual complexity is indefinitely scalable in geb
Alastair Channon · 2019
Closest in time.
Go-explore: a new approach for hard-exploration problems
Original
Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O Stanley, and Jeff Clune · 2019
Closest in time.
Imitating latent policies from observation
Ashley D Edwards, Himanshu Sahni, Yannick Schroeker, and Charles L Isbell · 2019
Closest in time.
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
Thomas Miconi, Aditya Rawal, Jeff Clune, and Kenneth O Stanley · 2019
Closest in time.
Generative teaching networks: learning to teach by generating synthetic training data
Felipe Petroski Such, Aditya Rawal, Joel Lehman, Kenneth O Stanley, and Jeff Clune · 2019
Closest in time.
Evolving images for visual neurons using a deep generative network reveals coding principles and neuronal preferences
Carlos R. Ponce, Will Xiao, Peter F. Schade, Till S. Hartmann, Gabriel Kreiman, and Margaret S. Livingstone · 2019
Closest in time.
Designing neural networks through neuroevolution
Kenneth O Stanley, Jeff Clune, Joel Lehman, and Risto Miikkulainen · 2019
Closest in time.
The bitter lesson, 2019
Rich Sutton · 2019
Closest in time.
Paired open-ended trailblazer (poet): Endlessly generating increasingly complex and diverse learning environments and their solutions
Original
Rui Wang, Joel Lehman, Jeff Clune, and Kenneth O Stanley · 2019
Closest in time.
Learning to continually learn
S. Beaulieu, L. Frati, T. Miconi, J. Lehman, K. Stanley, J. Clune, and N. Cheney · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…