2017

Recurrent Relational Networks

Palm, Rasmus Berg, Paquet, Ulrich, Winther, Ole

Understand

This paper is concerned with learning to solve tasks that require a chain of interdependent steps of relational inference, like answering complex questions about the relationships between objects, or solving puzzles where the smaller elements of a solution mutually constrain each other.

We introduce the recurrent relational network, a general purpose module that operates on a graph representation of objects.
As a generalization of Santoro et al.
[2017]'s relational network, it can augment any neural network model with the capacity to do many-step relational reasoning.

Built on

A logical calculus of the ideas immanent in nervous activity
Warren S McCulloch and Walter Pitts · 1943
Earlier work this paper cites.
The development of object perception
Elizabeth S Spelke, Grant Gutheil, and Gretchen Van de Walle · 1995
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Loopy belief propagation for approximate inference: An empirical study
Kevin P Murphy, Yair Weiss, and Michael I Jordan · 1999
Earlier work this paper cites.
Dancing links
Donald E Knuth · 2000
Earlier work this paper cites.
Solving every Sudoku puzzle, 2006
Peter Norvig · 2006
Earlier work this paper cites.
Core knowledge
Elizabeth S Spelke and Katherine D Kinzler · 2007
Earlier work this paper cites.
Passing messages to lonely numbers
Heiko Bauke · 2008
Earlier work this paper cites.
The graph neural network model
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini · 2009
Earlier work this paper cites.
Learning message-passing inference machines for structured prediction
Stephane Ross, Daniel Munoz, Martial Hebert, and J Andrew Bagnell · 2011
Earlier work this paper cites.
Learning to pass expectation propagation messages
Nicolas Heess, Daniel Tarlow, and John Winn · 2013
Earlier work this paper cites.

Similar

Solving Sudoku using probabilistic graphical models
Sheehan Khan, Shahab Jabbari, Shahin Jabbari, and Majid Ghanbarinejad · 2014
Cited alongside, same era.
Minimum Sudoku, 2014
Gordon Royle · 2014
Cited alongside, same era.
Deeply learning the messages in message passing inference
Guosheng Lin, Chunhua Shen, Ian Reid, and Anton van den Hengel · 2015
Cited alongside, same era.
Lifted relational neural networks
Gustav Šourek, Vojtech Aschenbrenner, Filip Železny, and Ondřej Kuželka · 2015
Cited alongside, same era.
End-to-end memory networks
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al · 2015
Cited alongside, same era.
Towards AI-complete question answering: A set of prerequisite toy tasks
Original
Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov · 2015
Cited alongside, same era.
Interaction networks for learning about objects, relations and physics
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al · 2016
Cited alongside, same era.
Structure inference machines: Recurrent neural networks for analyzing relations in group activity recognition
Zhiwei Deng, Arash Vahdat, Hexiang Hu, and Greg Mori · 2016
Cited alongside, same era.
Tracking the world state with recurrent entity networks
Original
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun · 2016
Cited alongside, same era.
Building machines that learn and think like people
Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman · 2016
Cited alongside, same era.
Can neural networks crack Sudoku?, 2016
Kyubyong Park · 2016
Cited alongside, same era.

Then

Scaling memory-augmented neural networks with sparse reads and writes
Jack Rae, Jonathan J Hunt, Ivo Danihelka, Timothy Harley, Andrew W Senior, Gregory Wayne, Alex Graves, and Tim Lillicrap · 2016
Later among the works it cites.
Statistical relational artificial intelligence: Logic, probability, and computation
Luc De Raedt, Kristian Kersting, Sriraam Natarajan, and David Poole · 2016
Later among the works it cites.
Learning and reasoning with logic tensor networks
Luciano Serafini and Artur S d’Avila Garcez · 2016
Later among the works it cites.
Learning multiagent communication with backpropagation
Sainbayar Sukhbaatar, Rob Fergus, et al · 2016
Later among the works it cites.
Convolutional pose machines
Shih-En Wei, Varun Ramakrishna, Takeo Kanade, and Yaser Sheikh · 2016
Later among the works it cites.
Optnet: Differentiable optimization as a layer in neural networks
Original
Brandon Amos and J Zico Kolter · 2017
Closest in time.
Neural-symbolic learning and reasoning: A survey and interpretation
Original
Tarek R Besold, Artur d’Avila Garcez, Sebastian Bader, Howard Bowman, Pedro Domingos, Pascal Hitzler, Kai-Uwe Kühnberger, Luis C Lamb, Daniel Lowd, Priscila Machado Vieira Lima, et al · 2017
Closest in time.
Neural message passing for quantum chemistry
Original
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl · 2017
Closest in time.
A simple neural network module for relational reasoning
Original
Adam Santoro, David Raposo, David GT Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap · 2017
Closest in time.
Finding remo (related memory object): A simple neural architecture for text based reasoning
Original
Hyochang Yang, Sungzoon Cho, et al · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…