2019

Contrastive Learning of Structured World Models

Kipf, Thomas, van der Pol, Elise, Welling, Max

Understand

A structured understanding of our world in terms of objects, relations, and hierarchies is an important component of human cognition.

Learning such a structured world model from raw sensory data remains a challenge.
As a step towards this goal, we introduce Contrastively-trained Structured World Models (C-SWMs).
C-SWMs utilize a contrastive approach for representation learning in environments with compositional structure.

Built on

Relational action forecasting
Original
Chen Sun, Abhinav Shrivastava, Carl Vondrick, Rahul Sukthankar, Kevin Murphy, and Cordelia Schmid · 1904
Earlier work this paper cites.
Contrastive bidirectional transformer for temporal representation learning
Original
Chen Sun, Fabien Baradel, Kevin Murphy, and Cordelia Schmid · 1906
Earlier work this paper cites.
Changing views of Attention and Automaticity
Daniel Kahneman and Anne Treisman · 1984
Earlier work this paper cites.
The reviewing of object files: Object-specific integration of information
Daniel Kahneman, Anne Treisman, and Brian J Gibbs · 1992
Earlier work this paper cites.
Self-improving reactive agents based on reinforcement learning, planning and teaching
Long-Ji Lin · 1992
Earlier work this paper cites.
A tutorial on energy-based learning
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang · 2006
Earlier work this paper cites.
Matplotlib: A 2d graphics environment
John D Hunter · 2007
Earlier work this paper cites.
Core knowledge
Elizabeth S Spelke and Katherine D Kinzler · 2007
Earlier work this paper cites.
The graph neural network model
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini · 2009
Earlier work this paper cites.
A three-way model for collective learning on multi-relational data
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel · 2011
Earlier work this paper cites.
A fast and simple algorithm for training neural probabilistic language models
Original
Andriy Mnih and Yee Whye Teh · 2012
Earlier work this paper cites.
The arcade learning environment: An evaluation platform for general agents
Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling · 2013
Earlier work this paper cites.
Translating embeddings for modeling multi-relational data
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko · 2013
Earlier work this paper cites.
Auto-encoding variational bayes
Original
Diederik P Kingma and Max Welling · 2013
Earlier work this paper cites.
Efficient estimation of word representations in vector space
Original
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Deepwalk: Online learning of social representations
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena · 2014
Earlier work this paper cites.
Stochastic backpropagation and approximate inference in deep generative models
Original
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra · 2014
Earlier work this paper cites.
Knowledge graph embedding by translating on hyperplanes
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen · 2014
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Original
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Learning state representations with robotic priors
Rico Jonschkowski and Oliver Brock · 2015
Earlier work this paper cites.
Gated graph sequence neural networks
Original
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel · 2015
Earlier work this paper cites.
Embed to control: A locally linear latent dynamics model for control from raw images
Manuel Watter, Jost Springenberg, Joschka Boedecker, and Martin Riedmiller · 2015
Earlier work this paper cites.
Empirical evaluation of rectified activations in convolutional network
Original
Bing Xu, Naiyan Wang, Tianqi Chen, and Mu Li · 2015
Earlier work this paper cites.

Similar

Layer normalization
Original
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton · 2016
Cited alongside, same era.
Interaction networks for learning about objects, relations and physics
Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al · 2016
Cited alongside, same era.
Openai gym
Original
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba · 2016
Cited alongside, same era.
A compositional object-based approach to learning physical dynamics
Original
Michael B Chang, Tomer Ullman, Antonio Torralba, and Joshua B Tenenbaum · 2016
Cited alongside, same era.
Attend, infer, repeat: Fast scene understanding with generative models
SM Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, David Szepesvari, Koray Kavukcuoglu, and Geoffrey E Hinton · 2016
Cited alongside, same era.
node2vec: Scalable feature learning for networks
Aditya Grover and Jure Leskovec · 2016
Cited alongside, same era.
Semi-supervised classification with graph convolutional networks
Original
Thomas N Kipf and Max Welling · 2016
Cited alongside, same era.
Learning multiagent communication with backpropagation
Sainbayar Sukhbaatar, Rob Fergus, et al · 2016
Cited alongside, same era.
Complex embeddings for simple link prediction
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard · 2016
Cited alongside, same era.
Neural message passing for quantum chemistry
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl · 2017
Cited alongside, same era.
Neural expectation maximization
Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber · 2017
Cited alongside, same era.
Vain: Attentional multi-agent predictive modeling
Yedid Hoshen · 2017
Cited alongside, same era.

Then

Curiosity driven exploration of learned disentangled goal spaces
Original
Adrien Laversanne-Finot, Alexandre Péré, and Pierre-Yves Oudeyer · 2018
Later among the works it cites.
Representation learning with contrastive predictive coding
Original
Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2018
Later among the works it cites.
Graph networks as learnable physics engines for inference and control
Alvaro Sanchez-Gonzalez, Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, and Peter Battaglia · 2018
Later among the works it cites.
Modeling relational data with graph convolutional networks
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling · 2018
Later among the works it cites.
Actor-centric relation network
Chen Sun, Abhinav Shrivastava, Carl Vondrick, Kevin Murphy, Rahul Sukthankar, and Cordelia Schmid · 2018
Later among the works it cites.
Disentangling the independently controllable factors of variation by interacting with the world
Original
Valentin Thomas, Emmanuel Bengio, William Fedus, Jules Pondard, Philippe Beaudoin, Hugo Larochelle, Joelle Pineau, Doina Precup, and Yoshua Bengio · 2018
Later among the works it cites.
Relational neural expectation maximization: Unsupervised discovery of objects and their interactions
Original
Sjoerd van Steenkiste, Michael Chang, Klaus Greff, and Jürgen Schmidhuber · 2018
Later among the works it cites.
Deep graph infomax
Original
Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm · 2018
Later among the works it cites.
Nervenet: Learning structured policy with graph neural networks
Tingwu Wang, Renjie Liao, Jimmy Ba, and Sanja Fidler · 2018
Later among the works it cites.
Unsupervised state representation learning in Atari
Original
Ankesh Anand, Evan Racah, Sherjil Ozair, Yoshua Bengio, Marc-Alexandre Côté, and R Devon Hjelm · 2019
Closest in time.
Monet: Unsupervised scene decomposition and representation
Original
Christopher P Burgess, Loic Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, and Alexander Lerchner · 2019
Closest in time.
Genesis: Generative scene inference and sampling with object-centric latent representations
Original
Martin Engelcke, Adam R Kosiorek, Oiwi Parker Jones, and Ingmar Posner · 2019
Closest in time.
DeepMDP: Learning continuous latent space models for representation learning
Original
Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, and Marc G Bellemare · 2019
Closest in time.
Multi-object representation learning with iterative variational inference
Original
Klaus Greff, Raphaël Lopez Kaufmann, Rishab Kabra, Nick Watters, Chris Burgess, Daniel Zoran, Loic Matthey, Matthew Botvinick, and Alexander Lerchner · 2019
Closest in time.
Learning latent dynamics for planning from pixels
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson · 2019
Closest in time.
Data-efficient image recognition with contrastive predictive coding
Original
Olivier J Hénaff, Ali Razavi, Carl Doersch, SM Eslami, and Aaron van den Oord · 2019
Closest in time.
Reasoning about physical interactions with object-oriented prediction and planning
Michael Janner, Sergey Levine, William T Freeman, Joshua B Tenenbaum, Chelsea Finn, and Jiajun Wu · 2019
Closest in time.
Physics-as-inverse-graphics: Joint unsupervised learning of objects and physics from video
Original
Miguel Jaques, Michael Burke, and Timothy Hospedales · 2019
Closest in time.
Model-based reinforcement learning for atari
Original
Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, et al · 2019
Closest in time.
Compile: Compositional imitation learning and execution
Thomas Kipf, Yujia Li, Hanjun Dai, Vinicius Zambaldi, Alvaro Sanchez-Gonzalez, Edward Grefenstette, Pushmeet Kohli, and Peter Battaglia · 2019
Closest in time.
Learning robotic manipulation through visual planning and acting
Original
Angelina Wang, Thanard Kurutach, Kara Liu, Pieter Abbeel, and Aviv Tamar · 2019
Closest in time.
Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration
Original
Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher P Burgess, and Alexander Lerchner · 2019
Closest in time.
Unsupervised discovery of parts, structure, and dynamics
Original
Zhenjia Xu, Zhijian Liu, Chen Sun, Kevin Murphy, William T Freeman, Joshua B Tenenbaum, and Jiajun Wu · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…