2019

ROBEL: Robotics Benchmarks for Learning with Low-Cost Robots

Ahn, Michael, Zhu, Henry, Hartikainen, Kristian et al.

Understand

ROBEL is an open-source platform of cost-effective robots designed for reinforcement learning in the real world.

ROBEL introduces two robots, each aimed to accelerate reinforcement learning research in different task domains: D'Claw is a three-fingered hand robot that facilitates learning dexterous manipulation tasks, and D'Kitty is a four-legged robot that facilitates learning agile legged locomotion tasks.
These low-cost, modular robots are easy to maintain and are robust enough to sustain on-hardware reinforcement learning from scratch with over 14000 training hours registered on them to date.
To leverage this platform, we propose an extensible set of continuous control benchmark tasks for each robot.

Built on

Active domain randomization
Original
M. Bhairav, D. Manfred, G. Florian, J. P. Christopher, and P. Liam · 1904
Earlier work this paper cites.
Markov decision processes: Discrete stochastic dynamic programming
M. L. Puterman · 1994
Earlier work this paper cites.
A three-dimensional passive-dynamic walking robot with two legs and knees
S. H. Collins, M. Wisse, and A. Ruina · 2001
Earlier work this paper cites.
A natural policy gradient
S. M. Kakade · 2002
Earlier work this paper cites.
Robot competitions-ideal benchmarks for robotics research
S. Behnke · 2006
Earlier work this paper cites.
Stanley: The robot that won the darpa grand challenge
S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron, J. Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann, et al · 2006
Earlier work this paper cites.
Towards a personal robotics development platform: Rationale and design of an intrinsically safe personal robot
K. A. Wyrobek, E. H. Berger, H. M. Van der Loos, and J. K. Salisbury · 2008
Earlier work this paper cites.
The raven: Design and validation of a telesurgery system
M. J. Lum, D. C. Friedman, G. Sankaranarayanan, H. King, K. Fodero, R. Leuschke, B. Hannaford, J. Rosen, and M. N. Sinanan · 2009
Earlier work this paper cites.
The highly adaptive sdm hand: Design and performance evaluation
A. M. Dollar and R. D. Howe · 2010
Earlier work this paper cites.
Nonlinear model predictive control , volume 26
F. Allgöwer and A. Zheng · 2012
Earlier work this paper cites.
Mujoco: A physics engine for model-based control
E. Todorov, T. Erez, and Y. Tassa · 2012
Earlier work this paper cites.
Continuous control with deep reinforcement learning
Original
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra · 2015
Earlier work this paper cites.
Team ihmc’s lessons learned from the darpa robotics challenge trials
M. Johnson, B. Shrewsbury, S. Bertrand, T. Wu, D. Duran, M. Floyd, P. Abeles, D. Stephen, N. Mertins, A. Lesman, et al · 2015
Earlier work this paper cites.
Design and fabrication of a soft robotic hand with embedded actuators and sensors
Y. She, C. Li, J. Cleary, and H.-J. Su · 2015
Earlier work this paper cites.

Similar

The mit super mini cheetah: A small, low-cost quadrupedal robot for dynamic locomotion
W. Bosworth, S. Kim, and N. Hogan · 2015
Cited alongside, same era.
The ycb object and model set: Towards common benchmarks for manipulation research
B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar · 2015
Cited alongside, same era.
Openai gym
Original
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba · 2016
Cited alongside, same era.
Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours
L. Pinto and A. Gupta · 2016
Cited alongside, same era.
Benchmarking deep reinforcement learning for continuous control
Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel · 2016
Cited alongside, same era.
Cad2rl: Real single-image flight without a single real image
Original
F. Sadeghi and S. Levine · 2016
Cited alongside, same era.
Design of a highly biomimetic anthropomorphic robotic hand towards artificial limb regeneration
Z. Xu and E. Todorov · 2016
Cited alongside, same era.
Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning
X. B. Peng, G. Berseth, K. Yin, and M. Van De Panne · 2017
Cited alongside, same era.
Domain randomization for transferring deep neural networks from simulation to the real world
J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel · 2017
Cited alongside, same era.
The robotarium: A remotely accessible swarm robotics research testbed
D. Pickem, P. Glotfelter, L. Wang, M. Mote, A. Ames, E. Feron, and M. Egerstedt · 2017
Cited alongside, same era.
Learning complex dexterous manipulation with deep reinforcement learning and demonstrations
Original
A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine · 2017
Cited alongside, same era.
Deepmind control suite
Original
Y. Tassa, Y. Doron, A. Muldal, T. Erez, Y. Li, D. d. L. Casas, D. Budden, A. Abdolmaleki, J. Merel, A. Lefrancq, et al · 2018
Cited alongside, same era.

Then

Reinforcement learning: An introduction
R. S. Sutton and A. G. Barto · 2018
Later among the works it cites.
Soft actor-critic algorithms and applications
Original
T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, et al · 2018
Later among the works it cites.
Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection
S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen · 2018
Later among the works it cites.
Dexterous manipulation with deep reinforcement learning: Efficient, general, and low-cost
Original
H. Zhu, A. Gupta, A. Rajeswaran, S. Levine, and V. Kumar · 2018
Later among the works it cites.
Learning dexterous in-hand manipulation
Original
M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, et al · 2018
Later among the works it cites.
Sim-to-real reinforcement learning for deformable object manipulation
Original
J. Matas, S. James, and A. J. Davison · 2018
Later among the works it cites.
Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation
D. Kalashnikov, A. Irpan, P. P. Sampedro, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, and S. Levine · 2018
Later among the works it cites.
Benchmarking reinforcement learning algorithms on real-world robots
Original
A. R. Mahmood, D. Korenkevych, G. Vasan, W. Ma, and J. Bergstra · 2018
Later among the works it cites.
Bayessim: adaptive domain randomization via probabilistic inference for robotics simulators
Original
F. Ramos, R. C. Possas, and D. Fox · 2019
Closest in time.
Replab: A reproducible low-cost arm benchmark platform for robotic learning
Original
B. Yang, J. Zhang, V. Pong, S. Levine, and D. Jayaraman · 2019
Closest in time.
http://www.robotis.us/dynamixel/
Dynamixel smart actuator · 2019
Closest in time.
http://www.robotis.us/u2d2/
Usb-serial bus · 2019
Closest in time.
Stochastic latent actor-critic: Deep reinforcement learning with a latent variable model
Original
A. X. Lee, A. Nagabandi, P. Abbeel, and S. Levine · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…