Fetching the paper…

Reading the bibliography…

2017

Sim-to-Real Transfer of Robotic Control with Dynamics Randomization

Peng, Xue Bin, Andrychowicz, Marcin, Zaremba, Wojciech et al.

Original
PDF
Post on X

Understand

Simulations are attractive environments for training agents as they provide an abundant source of data and alleviate certain safety concerns during the training process.

But the behaviours developed by agents in simulation are often specific to the characteristics of the simulator.
Due to modeling error, strategies that are successful in simulation may not transfer to their real world counterparts.
In this paper, we demonstrate a simple method to bridge this "reality gap".

Sim-to-Real Transfer of Robotic Control with Dynamics Randomization · Around

Built on

K. M. Lynch and M. T. Mason, “Stable pushing: Mechanics, controllability, and planning,” The International Journal of Robotics Research , vol. 15, no. 6, pp. 533–556, 1996
1996
Earlier work this paper cites.
S. Akella and M. T. Mason, “Posing polygonal objects in the plane by pushing,” The International Journal of Robotics Research , vol. 17, no. 1, pp. 70–88, 1998
1998
Earlier work this paper cites.
R. S. Sutton, D. Mcallester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” in In Advances in Neural Information Processing Systems 12 . MIT Press, 2000, pp. 1057–1063
2000
Earlier work this paper cites.
M. Dogar and S. Srinivasa, “A framework for push-grasping in clutter,” in Robotics: Science and Systems VII . Pittsburgh, PA: MIT Press, July 2011
2011
Earlier work this paper cites.
E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A physics engine for model-based control.” in IROS . IEEE, 2012, pp. 5026–5033
2012
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR , vol. abs/1412.6980, 2014. [Online]. Available: http://arxiv.org/abs/1412.6980
Original
2014
Earlier work this paper cites.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature , vol. 518, no. 7540, pp. 529–533, 02 2015. [Online]. Available: http://dx.doi.org/10.1038/nature14236
2015
Earlier work this paper cites.
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” CoRR , vol. abs/1509.02971, 2015. [Online]. Available: http://arxiv.org/abs/1509.02971
Original
2015
Earlier work this paper cites.
S. Levine, N. Wagener, and P. Abbeel, “Learning contact-rich manipulation skills with guided policy search,” CoRR , vol. abs/1501.05611, 2015. [Online]. Available: http://arxiv.org/abs/1501.05611
Original
2015
Earlier work this paper cites.
S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” CoRR , vol. abs/1504.00702, 2015. [Online]. Available: http://arxiv.org/abs/1504.00702
Original
2015
Earlier work this paper cites.
E. Tzeng, C. Devin, J. Hoffman, C. Finn, X. Peng, S. Levine, K. Saenko, and T. Darrell, “Adapting deep visuomotor representations with weak pairwise constraints,” CoRR , vol. abs/1511.07111, 2015. [Online]. Available: http://arxiv.org/abs/1511.07111
Original
2015
Earlier work this paper cites.
I. Mordatch, K. Lowrey, and E. Todorov, “Ensemble-cio: Full-body dynamic motion planning that transfers to physical humanoids,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2015, Hamburg, Germany, September 28 - October 2, 2015 , 2015, pp. 5307–5314. [Online]. Available: https://doi.org/10.1109/IROS.2015.7354126
2015
Earlier work this paper cites.
T. Schaul, D. Horgan, K. Gregor, and D. Silver, “Universal value function approximators,” in Proceedings of the 32nd International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, F. Bach and D. Blei, Eds., vol. 37. Lille, France: PMLR, 07–09 Jul 2015, pp. 1312–1320. [Online]. Available: http://proceedings.mlr.press/v37/schaul15.html
2015
Earlier work this paper cites.
N. Heess, J. J. Hunt, T. P. Lillicrap, and D. Silver, “Memory-based control with recurrent neural networks,” CoRR , vol. abs/1512.04455, 2015. [Online]. Available: http://arxiv.org/abs/1512.04455
Original
2015
Earlier work this paper cites.

Similar

Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep reinforcement learning for continuous control,” CoRR , vol. abs/1604.06778, 2016. [Online]. Available: http://arxiv.org/abs/1604.06778
Original
2016
Cited alongside, same era.
X. B. Peng, G. Berseth, and M. van de Panne, “Terrain-adaptive locomotion skills using deep reinforcement learning,” ACM Transactions on Graphics (Proc. SIGGRAPH 2016) , vol. 35, no. 4, 2016
2016
Cited alongside, same era.
S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” CoRR , vol. abs/1603.02199, 2016. [Online]. Available: http://arxiv.org/abs/1603.02199
Original
2016
Cited alongside, same era.
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” J. Mach. Learn. Res. , vol. 17, no. 1, pp. 2096–2030, Jan. 2016. [Online]. Available: http://dl.acm.org/citation.cfm?id=2946645.2946704

Then

J. N. Foerster, Y. M. Assael, N. de Freitas, and S. Whiteson, “Learning to communicate with deep multi-agent reinforcement learning,” CoRR , vol. abs/1605.06676, 2016. [Online]. Available: http://arxiv.org/abs/1605.06676
Original
2016
Later among the works it cites.
X. B. Peng, G. Berseth, K. Yin, and M. van de Panne, “Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning,” ACM Transactions on Graphics (Proc. SIGGRAPH 2017) , vol. 36, no. 4, 2017
2017
Closest in time.
L. Liu and J. Hodgins, “Learning to schedule control fragments for physics-based characters using deep q-learning,” ACM Trans. Graph. , vol. 36, no. 3, pp. 29:1–29:14, Jun. 2017. [Online]. Available: http://doi.acm.org/10.1145/3083723
2017
Closest in time.
N. Heess, D. TB, S. Sriram, J. Lemmon, J. Merel, G. Wayne, Y. Tassa, T. Erez, Z. Wang, S. M. A. Eslami, M. A. Riedmiller, and D. Silver, “Emergence of locomotion behaviours in rich environments,” CoRR , vol. abs/1707.02286, 2017. [Online]. Available: http://arxiv.org/abs/1707.02286
Original
2017

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2016

Cited alongside, same era.

S. Daftry, J. A. Bagnell, and M. Hebert, “Learning transferable policies for monocular reactive MAV control,” CoRR , vol. abs/1608.00627, 2016. [Online]. Available: http://arxiv.org/abs/1608.00627

2016

Cited alongside, same era.

A. A. Rusu, M. Vecerik, T. Rothörl, N. Heess, R. Pascanu, and R. Hadsell, “Sim-to-real robot learning from pixels with progressive nets,” CoRR , vol. abs/1610.04286, 2016. [Online]. Available: http://arxiv.org/abs/1610.04286

2016

Cited alongside, same era.

P. Christiano, Z. Shah, I. Mordatch, J. Schneider, T. Blackwell, J. Tobin, P. Abbeel, and W. Zaremba, “Transfer from simulation to real world through learning deep inverse dynamics model,” CoRR , vol. abs/1610.03518, 2016. [Online]. Available: http://arxiv.org/abs/1610.03518

2016

Cited alongside, same era.

F. Sadeghi and S. Levine, “Cad2rl: Real single-image flight without a single real image,” CoRR , vol. abs/1611.04201, 2016. [Online]. Available: http://arxiv.org/abs/1611.04201

2016

Cited alongside, same era.

S. James and E. Johns, “3d simulation for robot arm control with deep q-learning,” CoRR , vol. abs/1609.03759, 2016. [Online]. Available: http://arxiv.org/abs/1609.03759

2016

Cited alongside, same era.

A. Rajeswaran, S. Ghotra, S. Levine, and B. Ravindran, “Epopt: Learning robust neural network policies using model ensembles,” CoRR , vol. abs/1610.01283, 2016. [Online]. Available: http://arxiv.org/abs/1610.01283

2016

Cited alongside, same era.

K. Yu, M. Bauzá, N. Fazeli, and A. Rodriguez, “More than a million ways to be pushed: A high-fidelity experimental data set of planar pushing,” CoRR , vol. abs/1604.04038, 2016. [Online]. Available: http://arxiv.org/abs/1604.04038

2016

Cited alongside, same era.

N. Fazeli, R. Kolbert, R. Tedrake, and A. Rodriguez, “Parameter and contact force estimation of planar rigid-bodies undergoing frictional contact,” The International Journal of Robotics Research , vol. 0, no. 0, p. 0278364917698749, 2016

2016

Cited alongside, same era.

Closest in time.

A. Gupta, C. Devin, Y. Liu, P. Abbeel, and S. Levine, “Learning invariant feature spaces to transfer skills with reinforcement learning,” CoRR , vol. abs/1703.02949, 2017. [Online]. Available: http://arxiv.org/abs/1703.02949

2017

Closest in time.

M. Wulfmeier, I. Posner, and P. Abbeel, “Mutual alignment transfer learning,” CoRR , vol. abs/1707.07907, 2017. [Online]. Available: http://arxiv.org/abs/1707.07907

2017

Closest in time.

J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” CoRR , vol. abs/1703.06907, 2017. [Online]. Available: http://arxiv.org/abs/1703.06907

2017

Closest in time.

L. Pinto, J. Davidson, R. Sukthankar, and A. Gupta, “Robust adversarial reinforcement learning,” CoRR , vol. abs/1703.02702, 2017. [Online]. Available: http://arxiv.org/abs/1703.02702

2017

Closest in time.

W. Yu, C. K. Liu, and G. Turk, “Preparing for the unknown: Learning a universal policy with online system identification,” CoRR , vol. abs/1702.02453, 2017. [Online]. Available: http://arxiv.org/abs/1702.02453

2017

Closest in time.

R. Antonova, S. Cruciani, C. Smith, and D. Kragic, “Reinforcement learning for pivoting task,” CoRR , vol. abs/1703.00472, 2017. [Online]. Available: http://arxiv.org/abs/1703.00472

2017

Closest in time.

D. H. Ignasi Clavera and P. Abbeel, “Policy transfer via modularity,” in IROS . IEEE, 2017

2017

Closest in time.

M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, P. Abbeel, and W. Zaremba, “Hindsight experience replay,” in Advances in Neural Information Processing Systems , 2017

2017

Closest in time.

R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” CoRR , vol. abs/1706.02275, 2017. [Online]. Available: http://arxiv.org/abs/1706.02275

2017

Closest in time.