Built on

A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE transactions on systems, man, and cybernetics , no. 5, pp. 834–846, 1983
1983
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on . IEEE, 2009, pp. 248–255
2009
Earlier work this paper cites.
E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on . IEEE, 2012, pp. 5026–5033
2012
Earlier work this paper cites.
A. G. Barto, “Intrinsic motivation and reinforcement learning,” in Intrinsically motivated learning in natural and artificial systems . Springer, 2013, pp. 17–47
Original
2013
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision . Springer, 2014, pp. 740–755
2014
Earlier work this paper cites.
J. Scholz, M. Levihn, C. Isbell, and D. Wingate, “A physics-based model prior for object-oriented mdps,” in International Conference on Machine Learning , 2014, pp. 1089–1097
2014
Earlier work this paper cites.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. , “Human-level control through deep reinforcement learning,” Nature , vol. 518, no. 7540, p. 529, 2015
2015
Earlier work this paper cites.
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971 , 2015
Original
2015
Earlier work this paper cites.
R. Jonschkowski and O. Brock, “Learning state representations with robotic priors,” Autonomous Robots , vol. 39, no. 3, pp. 407–428, 2015
2015
Earlier work this paper cites.
M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller, “Embed to control: A locally linear latent dynamics model for control from raw images,” in Advances in neural information processing systems , 2015, pp. 2746–2754
2015
Earlier work this paper cites.
D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Computer Vision (ICCV), 2015 IEEE International Conference on . IEEE, 2015, pp. 4489–4497
2015
Earlier work this paper cites.
J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 2625–2634
2015
Earlier work this paper cites.
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” CoRR , vol. abs/1503.03832, 2015
Original
2015
Earlier work this paper cites.
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540 , 2016
Original
2016
Earlier work this paper cites.

Similar

H. van Hoof, N. Chen, M. Karl, P. van der Smagt, and J. Peters, “Stable reinforcement learning with autoencoders for tactile and visual data,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on . IEEE, 2016, pp. 3928–3934
2016
Cited alongside, same era.
C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel, “Deep spatial autoencoders for visuomotor learning,” in Robotics and Automation (ICRA), 2016 IEEE International Conference on . IEEE, 2016, pp. 512–519
2016
Cited alongside, same era.
J. Munk, J. Kober, and R. Babuška, “Learning state representation for deep actor-critic control,” in Decision and Control (CDC), 2016 IEEE 55th Conference on . IEEE, 2016, pp. 4667–4673
2016
Cited alongside, same era.
P. Agrawal, A. V. Nair, P. Abbeel, J. Malik, and S. Levine, “Learning to poke by poking: Experiential learning of intuitive physics,” in Advances in Neural Information Processing Systems , 2016, pp. 5074–5082
2016
Cited alongside, same era.

Then

R. Jonschkowski, R. Hafner, J. Scholz, and M. Riedmiller, “Pves: Position-velocity encoders for unsupervised learning of structured state representations,” arXiv preprint arXiv:1705.09805 , 2017
Original
2017
Later among the works it cites.
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017
Original
2017
Later among the works it cites.
T. Lesort, M. Seurin, X. Li, N. D. Rodríguez, and D. Filliat, “Unsupervised state representation learning with robotic priors: a robustness benchmark,” arXiv preprint arXiv:1709.05185 , 2017
Original
2017
Later among the works it cites.
M. Babaeizadeh, C. Finn, D. Erhan, R. H. Campbell, and S. Levine, “Stochastic variational video prediction,” arXiv preprint arXiv:1710.11252 , 2017
Original
2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2018

Learning Actionable Representations from Visual Observations

Dwibedi, Debidatta, Tompson, Jonathan, Lynch, Corey et al.

Understand

In this work we explore a new approach for robots to teach themselves about the world simply by observing it.

In particular we investigate the effectiveness of learning task-agnostic representations for continuous control tasks.
We extend Time-Contrastive Networks (TCN) that learn from visual observations by embedding multiple frames jointly in the embedding space as opposed to a single frame.
We show that by doing so, we are now able to encode both position and velocity attributes significantly more accurately.

Built on

A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike adaptive elements that can solve difficult learning control problems,” IEEE transactions on systems, man, and cybernetics , no. 5, pp. 834–846, 1983
1983
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on . IEEE, 2009, pp. 248–255
2009
Earlier work this paper cites.
E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control,” in Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on . IEEE, 2012, pp. 5026–5033
2012
Earlier work this paper cites.
A. G. Barto, “Intrinsic motivation and reinforcement learning,” in Intrinsically motivated learning in natural and artificial systems . Springer, 2013, pp. 17–47
Original
2013
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision . Springer, 2014, pp. 740–755
2014
Earlier work this paper cites.
J. Scholz, M. Levihn, C. Isbell, and D. Wingate, “A physics-based model prior for object-oriented mdps,” in International Conference on Machine Learning , 2014, pp. 1089–1097
2014
Earlier work this paper cites.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. , “Human-level control through deep reinforcement learning,” Nature , vol. 518, no. 7540, p. 529, 2015
2015
Earlier work this paper cites.
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971 , 2015
Original
2015
Earlier work this paper cites.
R. Jonschkowski and O. Brock, “Learning state representations with robotic priors,” Autonomous Robots , vol. 39, no. 3, pp. 407–428, 2015
2015
Earlier work this paper cites.
M. Watter, J. Springenberg, J. Boedecker, and M. Riedmiller, “Embed to control: A locally linear latent dynamics model for control from raw images,” in Advances in neural information processing systems , 2015, pp. 2746–2754
2015
Earlier work this paper cites.
D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Computer Vision (ICCV), 2015 IEEE International Conference on . IEEE, 2015, pp. 4489–4497
2015
Earlier work this paper cites.
J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 2625–2634
2015
Earlier work this paper cites.
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” CoRR , vol. abs/1503.03832, 2015
Original
2015
Earlier work this paper cites.
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint arXiv:1606.01540 , 2016
Original
2016
Earlier work this paper cites.

Similar

H. van Hoof, N. Chen, M. Karl, P. van der Smagt, and J. Peters, “Stable reinforcement learning with autoencoders for tactile and visual data,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on . IEEE, 2016, pp. 3928–3934
2016
Cited alongside, same era.
C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel, “Deep spatial autoencoders for visuomotor learning,” in Robotics and Automation (ICRA), 2016 IEEE International Conference on . IEEE, 2016, pp. 512–519
2016
Cited alongside, same era.
J. Munk, J. Kober, and R. Babuška, “Learning state representation for deep actor-critic control,” in Decision and Control (CDC), 2016 IEEE 55th Conference on . IEEE, 2016, pp. 4667–4673
2016
Cited alongside, same era.
P. Agrawal, A. V. Nair, P. Abbeel, J. Malik, and S. Levine, “Learning to poke by poking: Experiential learning of intuitive physics,” in Advances in Neural Information Processing Systems , 2016, pp. 5074–5082
2016
Cited alongside, same era.

Then

R. Jonschkowski, R. Hafner, J. Scholz, and M. Riedmiller, “Pves: Position-velocity encoders for unsupervised learning of structured state representations,” arXiv preprint arXiv:1705.09805 , 2017
Original
2017
Later among the works it cites.
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017
Original
2017
Later among the works it cites.
T. Lesort, M. Seurin, X. Li, N. D. Rodríguez, and D. Filliat, “Unsupervised state representation learning with robotic priors: a robustness benchmark,” arXiv preprint arXiv:1709.05185 , 2017
Original
2017
Later among the works it cites.
M. Babaeizadeh, C. Finn, D. Erhan, R. H. Campbell, and S. Levine, “Stochastic variational video prediction,” arXiv preprint arXiv:1710.11252 , 2017
Original
2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

L. Pinto, D. Gandhi, Y. Han, Y.-L. Park, and A. Gupta, “The curious robot: Learning visual representations via physical interactions,” in European Conference on Computer Vision . Springer, 2016, pp. 3–18

2016

Cited alongside, same era.

C. Finn, I. Goodfellow, and S. Levine, “Unsupervised learning for physical interaction through video prediction,” in Advances in neural information processing systems , 2016, pp. 64–72

2016

Cited alongside, same era.

A. Dosovitskiy and V. Koltun, “Learning to act by predicting the future,” arXiv preprint arXiv:1611.01779 , 2016

Original

2016

Cited alongside, same era.

E. Shelhamer, P. Mahmoudieh, M. Argus, and T. Darrell, “Loss is its own reward: Self-supervision for reinforcement learning,” arXiv preprint arXiv:1612.07307 , 2016

Original

2016

Cited alongside, same era.

M. Jaderberg, V. Mnih, W. M. Czarnecki, T. Schaul, J. Z. Leibo, D. Silver, and K. Kavukcuoglu, “Reinforcement learning with unsupervised auxiliary tasks,” arXiv preprint arXiv:1611.05397 , 2016

Original

2016

Cited alongside, same era.

P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard, A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuoglu, et al. , “Learning to navigate in complex environments,” arXiv preprint arXiv:1611.03673 , 2016

Original

2016

Cited alongside, same era.

K. Sohn, “Improved deep metric learning with multi-class n-pair loss objective,” in Advances in Neural Information Processing Systems , 2016, pp. 1857–1865

2016

Cited alongside, same era.

I. Misra, C. L. Zitnick, and M. Hebert, “Shuffle and learn: unsupervised learning using temporal order verification,” in European Conference on Computer Vision . Springer, 2016, pp. 527–544

2016

Cited alongside, same era.

L. Pinto and A. Gupta, “Learning to push by grasping: Using multiple tasks for effective learning,” in Robotics and Automation (ICRA), 2017 IEEE International Conference on . IEEE, 2017, pp. 2161–2168

2017

Later among the works it cites.

Y. Teh, V. Bapst, W. M. Czarnecki, J. Quan, J. Kirkpatrick, R. Hadsell, N. Heess, and R. Pascanu, “Distral: Robust multitask reinforcement learning,” in Advances in Neural Information Processing Systems , 2017, pp. 4499–4509

2017

Later among the works it cites.

J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 2017, pp. 4724–4733

2017

Later among the works it cites.

P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, and S. Levine, “Time-contrastive networks: Self-supervised learning from video,” Proceedings of International Conference in Robotics and Automation (ICRA 2018)

2018

Closest in time.

Y. Tassa, Y. Doron, A. Muldal, T. Erez, Y. Li, D. d. L. Casas, D. Budden, A. Abdolmaleki, J. Merel, A. Lefrancq, et al. , “Deepmind control suite,” arXiv preprint arXiv:1801.00690 , 2018

Original

2018

Closest in time.

A. Amiranashvili, A. Dosovitskiy, V. Koltun, and T. Brox, “Motion perception in reinforcement learning with dynamic objects,” in Proceedings of The 2nd Conference on Robot Learning , ser. Proceedings of Machine Learning Research, A. Billard, A. Dragan, J. Peters, and J. Morimoto, Eds., vol. 87. PMLR, 29–31 Oct 2018, pp. 156–168. [Online]. Available: http://proceedings.mlr.press/v87/amiranashvili18a.html

2018

Closest in time.

T. Yu, C. Finn, A. Xie, S. Dasari, T. Zhang, P. Abbeel, and S. Levine, “One-shot imitation from observing humans via domain-adaptive meta-learning,” arXiv preprint arXiv:1802.01557 , 2018

Original

2018

Closest in time.

D. Pathak, P. Mahmoudieh, G. Luo, P. Agrawal, D. Chen, F. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell, “Zero-shot visual imitation,” in ICLR , 2018

2018

Closest in time.

H. Arora, R. Kumar, J. Krone, and C. Li, “Multi-task learning for continuous control,” arXiv preprint arXiv:1802.01034 , 2018

Original

2018

Closest in time.