Built on

R. S. Sutton, “Integrated architectures for learning, planning, and reacting based on approximating dynamic programming,” in Machine Learning Proceedings 1990 . Elsevier, 1990, pp. 216–224
1990
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems (NIPS) , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
K. M. Kitani, B. D. Ziebart, J. A. Bagnell, and M. Hebert, “Activity forecasting,” in ECCV , 2012
2012
Earlier work this paper cites.
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602 , 2013
Original
2013
Earlier work this paper cites.
J. Koutník, J. Schmidhuber, and F. Gomez, “Online evolution of deep convolutional network for vision-based reinforcement learning,” in International Conference on Simulation of Adaptive Behavior , 2014
2014
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” International Conference on Learning Representations (ICLR) , 2014
2014
Earlier work this paper cites.
N. Heess, G. Wayne, D. Silver, T. Lillicrap, T. Erez, and Y. Tassa, “Learning continuous control policies by stochastic value gradients,” in Advances in Neural Information Processing Systems (NIPS) , 2015
2015
Earlier work this paper cites.
N. Wahlström, T. B. Schön, and M. P. Deisenroth, “From pixels to torques: Policy learning with deep dynamical models,” arXiv preprint arXiv:1502.02251 , 2015
Original
2015
Earlier work this paper cites.
F. Zhang, J. Leitner, M. Milford, B. Upcroft, and P. Corke, “Towards vision-based deep reinforcement learning for robotic motion control,” arXiv preprint arXiv:1511.03791 , 2015
Original
2015
Earlier work this paper cites.
W. Böhmer, J. T. Springenberg, J. Boedecker, M. Riedmiller, and K. Obermayer, “Autonomous learning of state representations for control,” KI-Künstliche Intelligenz , 2015
2015
Earlier work this paper cites.
M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller, “Embed to control: A locally linear latent dynamics model for control from raw images,” in Advances in Neural Information Processing Systems (NIPS) , 2015
2015
Earlier work this paper cites.
I. Lenz, R. A. Knepper, and A. Saxena, “Deepmpc: Learning deep latent features for model predictive control.” in RSS , 2015
2015
Earlier work this paper cites.

Similar

E. Tzeng, C. Devin, J. Hoffman, C. Finn, X. Peng, S. Levine, K. Saenko, and T. Darrell, “Towards adapting deep visuomotor representations from simulated to real environments,” arXiv:1511.07111 , 2015
Original
2015
Cited alongside, same era.
S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” arXiv preprint arXiv:1603.02199 , 2016
Original
2016
Cited alongside, same era.
L. Pinto and A. Gupta, “Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours,” in ICRA , 2016
2016
Cited alongside, same era.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in ECCV , 2016
2016
Cited alongside, same era.
C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel, “Deep spatial autoencoders for visuomotor learning,” in ICRA , 2016

Then

T. Weber, S. Racanière, D. P. Reichert, L. Buesing, A. Guez, D. J. Rezende, A. P. Badia, O. Vinyals, N. Heess, Y. Li et al. , “Imagination-augmented agents for deep reinforcement learning,” Advances in Neural Information Processing Systems (NIPS) , 2017
2017
Later among the works it cites.
J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017
2017
Later among the works it cites.
N. Rhinehart and K. M. Kitani, “First-person activity forecasting with online inverse reinforcement learning,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017
2017
Later among the works it cites.
A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine, “Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning,” arXiv preprint arXiv:1708.02596 , 2017
Original
2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2018

Learning Real-World Robot Policies by Dreaming

Piergiovanni, AJ, Wu, Alan, Ryoo, Michael S.

Understand

Learning to control robots directly based on images is a primary challenge in robotics.

However, many existing reinforcement learning approaches require iteratively obtaining millions of robot samples to learn a policy, which can take significant time.
In this paper, we focus on learning a realistic world model capturing the dynamics of scene changes conditioned on robot actions.
Our dreaming model can emulate samples equivalent to a sequence of images from the actual environment, technically by learning an action-conditioned future representation/scene regressor.

Built on

R. S. Sutton, “Integrated architectures for learning, planning, and reacting based on approximating dynamic programming,” in Machine Learning Proceedings 1990 . Elsevier, 1990, pp. 216–224
1990
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems (NIPS) , 2012, pp. 1097–1105
2012
Earlier work this paper cites.
K. M. Kitani, B. D. Ziebart, J. A. Bagnell, and M. Hebert, “Activity forecasting,” in ECCV , 2012
2012
Earlier work this paper cites.
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602 , 2013
Original
2013
Earlier work this paper cites.
J. Koutník, J. Schmidhuber, and F. Gomez, “Online evolution of deep convolutional network for vision-based reinforcement learning,” in International Conference on Simulation of Adaptive Behavior , 2014
2014
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” International Conference on Learning Representations (ICLR) , 2014
2014
Earlier work this paper cites.
N. Heess, G. Wayne, D. Silver, T. Lillicrap, T. Erez, and Y. Tassa, “Learning continuous control policies by stochastic value gradients,” in Advances in Neural Information Processing Systems (NIPS) , 2015
2015
Earlier work this paper cites.
N. Wahlström, T. B. Schön, and M. P. Deisenroth, “From pixels to torques: Policy learning with deep dynamical models,” arXiv preprint arXiv:1502.02251 , 2015
Original
2015
Earlier work this paper cites.
F. Zhang, J. Leitner, M. Milford, B. Upcroft, and P. Corke, “Towards vision-based deep reinforcement learning for robotic motion control,” arXiv preprint arXiv:1511.03791 , 2015
Original
2015
Earlier work this paper cites.
W. Böhmer, J. T. Springenberg, J. Boedecker, M. Riedmiller, and K. Obermayer, “Autonomous learning of state representations for control,” KI-Künstliche Intelligenz , 2015
2015
Earlier work this paper cites.
M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller, “Embed to control: A locally linear latent dynamics model for control from raw images,” in Advances in Neural Information Processing Systems (NIPS) , 2015
2015
Earlier work this paper cites.
I. Lenz, R. A. Knepper, and A. Saxena, “Deepmpc: Learning deep latent features for model predictive control.” in RSS , 2015
2015
Earlier work this paper cites.

Similar

E. Tzeng, C. Devin, J. Hoffman, C. Finn, X. Peng, S. Levine, K. Saenko, and T. Darrell, “Towards adapting deep visuomotor representations from simulated to real environments,” arXiv:1511.07111 , 2015
Original
2015
Cited alongside, same era.
S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” arXiv preprint arXiv:1603.02199 , 2016
Original
2016
Cited alongside, same era.
L. Pinto and A. Gupta, “Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours,” in ICRA , 2016
2016
Cited alongside, same era.
W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in ECCV , 2016
2016
Cited alongside, same era.
C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel, “Deep spatial autoencoders for visuomotor learning,” in ICRA , 2016

Then

T. Weber, S. Racanière, D. P. Reichert, L. Buesing, A. Guez, D. J. Rezende, A. P. Badia, O. Vinyals, N. Heess, Y. Li et al. , “Imagination-augmented agents for deep reinforcement learning,” Advances in Neural Information Processing Systems (NIPS) , 2017
2017
Later among the works it cites.
J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017
2017
Later among the works it cites.
N. Rhinehart and K. M. Kitani, “First-person activity forecasting with online inverse reinforcement learning,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017
2017
Later among the works it cites.
A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine, “Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning,” arXiv preprint arXiv:1708.02596 , 2017
Original
2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2016

Cited alongside, same era.

C. Finn, I. Goodfellow, and S. Levine, “Unsupervised learning for physical interaction through video prediction,” in Advances in Neural Information Processing Systems (NIPS) , 2016, pp. 64–72

2016

Cited alongside, same era.

T. Xue, J. Wu, K. L. Bouman, and W. T. Freeman, “Visual dynamics: Probabilistic future frame synthesis via cross convolutional networks,” in Advances in Neural Information Processing Systems (NIPS) , 2016

2016

Cited alongside, same era.

S. Gu, T. Lillicrap, I. Sutskever, and S. Levine, “Continuous deep q-learning with model-based acceleration,” in International Conference on Machine Learning (ICML) , 2016, pp. 2829–2838

2016

Cited alongside, same era.

A. Venkatraman, R. Capobianco, L. Pinto, M. Hebert, D. Nardi, and J. A. Bagnell, “Improved learning of dynamics models for control,” in International Symposium on Experimental Robotics , 2016

2016

Cited alongside, same era.

D. Silver, H. van Hasselt, M. Hessel, T. Schaul, A. Guez, T. Harley, G. Dulac-Arnold, D. Reichert, N. Rabinowitz, A. Barreto et al. , “The predictron: End-to-end learning and planning,” International Conference on Machine Learning (ICML) , 2016

2016

Cited alongside, same era.

P. Christiano, Z. Shah, I. Mordatch, J. Schneider, T. Blackwell, J. Tobin, P. Abbeel, and W. Zaremba, “Transfer from simulation to real world through learning deep inverse dynamics model,” arXiv preprint arXiv:1610.03518 , 2016

Original

2016

Cited alongside, same era.

C. Finn and S. Levine, “Deep visual foresight for planning robot motion,” in ICRA , 2017

2017

Cited alongside, same era.

J. Oh, S. Singh, and H. Lee, “Value prediction network,” Advances in Neural Information Processing Systems (NIPS) , 2017

2017

Later among the works it cites.

J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in IROS , 2017

2017

Later among the works it cites.

J. Lee and M. S. Ryoo, “Learning robot activities from first-person human videos using convolutional future regression,” in IROS , 2017

Original

2017

Later among the works it cites.

T. Lesort, N. Díaz-Rodríguez, and D. Filliat, “State representation learning for control: An overview,” Neural Networks , 2018

2018

Closest in time.

D. Pathak, P. Mahmoudieh, G. Luo, P. Agrawal, D. Chen, Y. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell, “Zero-shot visual imitation,” ICLR , 2018

2018

Closest in time.

V. Pong, S. Gu, M. Dalal, and S. Levine, “Temporal difference models: Model-free deep rl for model-based control,” arXiv preprint arXiv:1802.09081 , 2018

Original

2018

Closest in time.

D. Ha and J. Schmidhuber, “World models,” arXiv preprint arXiv:1803.10122 , 2018

Original

2018

Closest in time.