Fetching the paper…

Reading the bibliography…

2022

Multi-View Dreaming: Multi-View World Model with Contrastive Learning

Kinose, Akira, Okada, Masashi, Okumura, Ryo et al.

Original
PDF
Post on X

Understand

In this paper, we propose Multi-View Dreaming, a novel reinforcement learning agent for integrated recognition and control from multi-view observations by extending Dreaming.

Most current reinforcement learning method assumes a single-view observation space, and this imposes limitations on the observed data, such as lack of spatial information and occlusions.
This makes obtaining ideal observational information from the environment difficult and is a bottleneck for real-world robotics applications.
In this paper, we use contrastive learning to train a shared latent space between different viewpoints, and show how the Products of Experts approach can be used to integrate and control the probability distributions of latent states for multiple viewpoints.

Multi-View Dreaming: Multi-View World Model with Contrastive Learning · Around

Built on

L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,” Artificial intelligence
1998
Earlier work this paper cites.
M. Okada and T. Taniguchi, “Dreamingv2: Model-based reinforcement learning withdiscrete world models without reconstruction,” arXiv preprint arXiv:2203.00494
Original
2002
Earlier work this paper cites.
G. E. Hinton, “Training products of experts by minimizing contrastive divergence,” Neural computation
2002
Earlier work this paper cites.
P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, S. Levine, and G. Brain, “Time-contrastive networks: Self-supervised learning from video,” in International Conference on Robotics and Automation (ICRA)
2018
Earlier work this paper cites.
D. Dwibedi, J. Tompson, C. Lynch, and P. Sermanet, “Learning actionable representations from visual observations,” 2018
2018
Earlier work this paper cites.
D. Ha and J. Schmidhuber, “Recurrent world models facilitate policy evolution,” in Neural Information Processing Systems
2018
Earlier work this paper cites.
A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv:1807.03748
Original
2018
Earlier work this paper cites.
Y. Tassa, Y. Doron, A. Muldal, T. Erez, Y. Li, et al
Original
2018
Earlier work this paper cites.

Similar

M. Zhang, S. Vikram, L. Smith, P. Abbeel, M. Johnson, and S. Levine, “SOLAR: Deep structured representations for model-based reinforcement learning,” in International Conference on Machine Learning (ICML)
2019
Cited alongside, same era.
T. Kim, S. Ahn, and Y. Bengio, “Variational temporal abstraction,” Neural Information Processing Systems
2019
Cited alongside, same era.
A. X. Lee, A. Nagabandi, P. Abbeel, and S. Levine, “Stochastic latent actor-critic: Deep reinforcement learning with a latent variable model,” arXiv preprint arXiv:1907.00953
Original
2019
Cited alongside, same era.
D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning latent dynamics for planning from pixels,” in International Conference on Machine Learning (ICML)
2019
Cited alongside, same era.
M. Li, L. Wu, J. Wang, and H. Bou Ammar, “Multi-view reinforcement learning,” Neural Information Processing Systems

Then

A. Srinivas, M. Laskin, and P. Abbeel, “CURL: Contrastive unsupervised representations for reinforcement learning,” in International Conference on Machine Learning (ICML)
2020
Later among the works it cites.
W. Yan, A. Vangipuram, P. Abbeel, and L. Pinto, “Learning predictive representations for deformable objects using contrastive estimation,” arXiv:2003.05436
Original
2020
Later among the works it cites.
X. Ma, S. Chen, D. Hsu, and W. S. Lee, “Contrastive variational model-based reinforcement learning for complex observations,” Conference on Robot Learning (CoRL)
2020
Later among the works it cites.
Y. Zhu, J. Wong, A. Mandlekar, and R. Martín-Martín, “robosuite: A modular simulation framework and benchmark for robot learning,” arXiv:2009.12293
Original
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2019

Cited alongside, same era.

M. Okada, N. Kosaka, and T. Taniguchi, “PlaNet of the Bayesians: Reconsidering and improving deep planning network by incorporating Bayesian inference,” 2020

2020

Cited alongside, same era.

D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” International Conference on Learning Representations (ICLR)

2020

Cited alongside, same era.

T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International Conference on Learning Representations (ICLR)

2020

Cited alongside, same era.

M. A. Lee, Y. Zhu, K. Srinivasan, P. Shah, S. Savarese, L. Fei-Fei, A. Garg, and J. Bohg, “Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks,” in International Conference on Robotics and Automation (ICRA)

Cited in the paper.

K. Chen, Y. Lee, and H. Soh, “Multi-modal mutual information (MUMMI) training for robust self-supervised deep reinforcement learning,” in 2021 IEEE International Conference on Robotics and Automation (ICRA)

2021

Later among the works it cites.

M. Okada and T. Taniguchi, “Dreaming: Model-based reinforcement learning by latent imagination without reconstruction,” in International Conference on Robotics and Automation (ICRA)

2021

Later among the works it cites.

D. Hafner, T. Lillicrap, M. Norouzi, and J. Ba, “Mastering atari with discrete world models,” in International Conference on Learning Representations (ICLR)

2021

Later among the works it cites.

T. D. Nguyen, R. Shu, T. Pham, H. Bui, and S. Ermon, “Temporal predictive coding for model-based planning in latent space,” in International Conference on Machine Learning (ICML)

2021

Later among the works it cites.