2019

Learning to Make Analogies by Contrasting Abstract Relational Structure

Hill, Felix, Santoro, Adam, Barrett, David G. T. et al.

Understand

Analogical reasoning has been a principal focus of various waves of AI research.

Analogy is particularly challenging for machines because it requires relational structures to be represented such that they can be flexibly applied across diverse domains of experience.
Here, we study how analogical reasoning can be induced in neural networks that learn to perceive and reason about raw visual data.
We find that the critical factor for inducing such a capacity is not an elaborate architecture, but rather, careful attention to the choice of data and the manner in which it is presented to the model.

Built on

The problem of recognition
Mikhail M Bongard · 1967
Earlier work this paper cites.
The use of positive and negative examples during instruction
Ali M Ali · 1981
Earlier work this paper cites.
A computational model of analogical problem solving
Jaime G Carbonell · 1981
Earlier work this paper cites.
Structure-mapping: A theoretical framework for analogy
Dedre Gentner · 1983
Earlier work this paper cites.
Manual for raven’s progressive matrices and vocabulary scales
John C Raven · 1983
Earlier work this paper cites.
High-level perception, representation, and analogy: A critique of artificial intelligence methodology
David J Chalmers, Robert M French, and Douglas R Hofstadter · 1992
Earlier work this paper cites.
Analogy-making as perception: A computer model
Melanie Mitchell · 1993
Earlier work this paper cites.
Mental leaps
Keith James Holyoak and Paul Thagard · 1995
Earlier work this paper cites.
Fluid concepts and creative analogies
Douglas R Hofstadter · 1996
Earlier work this paper cites.
Distributed representations of structure: A theory of analogical access and mapping
John E Hummel and Keith J Holyoak · 1997
Earlier work this paper cites.
Analogy just looks like high level perception: Why a domain-general approach to analogical mapping is right
Kenneth D Forbus, Dedre Gentner, Arthur B Markman, and Ronald W Ferguson · 1998
Earlier work this paper cites.
Sex differences in spatial cognition, computational fluency, and arithmetical reasoning
David C Geary, Scott J Saults, Fan Liu, and Mary K Hoard · 2000
Earlier work this paper cites.
Working-memory modularity in analogical reasoning
Robert G Morrison, Keith J Holyoak, and Bao Troung · 2001
Earlier work this paper cites.

Similar

Cab: Connectionist analogy builder
Levi B Larkey and Bradley C Love · 2003
Cited alongside, same era.
Semantic cognition: A parallel distributed processing approach
Timothy T Rogers and James L McClelland · 2004
Cited alongside, same era.
Contrastive estimation: Training log-linear models on unlabeled data
Noah A. Smith and Jason Eisner · 2005
Cited alongside, same era.
The neural basis of analogical reasoning: An event-related potential study
Jiang Qiu, Hong Li, Antao Chen, and Qinglin Zhang · 2008
Cited alongside, same era.
Distance metric learning for large margin nearest neighbor classification
Kilian Q Weinberger and Lawrence K Saul · 2009
Cited alongside, same era.
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Michael Gutmann and Aapo Hyvärinen · 2010
Cited alongside, same era.
Compare & contrast: Teaching comparative thinking to strengthen student learning
Harvey F Silver · 2010
Cited alongside, same era.
Comparing machines and humans on a visual categorization test
François Fleuret, Ting Li, Charles Dubout, Emma K Wampler, Steven Yantis, and Donald Geman · 2011
Cited alongside, same era.
Computational models of analogy
Dedre Gentner and Kenneth D Forbus · 2011
Cited alongside, same era.
Efficient estimation of word representations in vector space
Original
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean · 2013
Cited alongside, same era.
Generative adversarial nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio · 2014
Cited alongside, same era.
A rational account of pedagogical reasoning: Teaching by, and learning from, examples
Patrick Shafto, Noah D Goodman, and Thomas L Griffiths · 2014
Cited alongside, same era.

Then

Hubness and pollution: Delving into cross-space mapping for zero-shot learning
Angeliki Lazaridou, Georgiana Dinu, and Marco Baroni · 2015
Later among the works it cites.
Deep visual analogy-making
Scott E Reed, Yi Zhang, Yuting Zhang, and Honglak Lee · 2015
Later among the works it cites.
Facenet: A unified embedding for face recognition and clustering
Florian Schroff, Dmitry Kalenichenko, and James Philbin · 2015
Later among the works it cites.
Meta-learning with memory-augmented neural networks
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap · 2016
Later among the works it cites.
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al · 2016
Later among the works it cites.
Model-agnostic meta-learning for fast adaptation of deep networks
Original
Chelsea Finn, Pieter Abbeel, and Sergey Levine · 2017
Later among the works it cites.
Modeling visual problem solving as analogical reasoning
Andrew Lovett and Kenneth Forbus · 2017
Later among the works it cites.
A simple neural network module for relational reasoning
Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Tim Lillicrap · 2017
Later among the works it cites.
Mastering the game of go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al · 2017
Later among the works it cites.
Generative adversarial active learning
Original
Jia-Jie Zhu and José Bento · 2017
Later among the works it cites.
Measuring abstract reasoning in neural networks
David Barrett, Felix Hill, Adam Santoro, Ari Morcos, and Timothy Lillicrap · 2018
Later among the works it cites.
Integrated deep visual and semantic attractor neural networks predict fmri pattern-information along the ventral object processing pathway
Barry J Devereux, Alex D Clarke, and Lorraine K Tyler · 2018
Later among the works it cites.
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
Brenden Lake and Marco Baroni · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…