Hierarchical Policy Design for Sample-Efficient Learning of Robot Table Tennis Through Self-Play

Understand

Training robots with physical bodies requires developing new methods and action representations that allow the learning agents to explore the space of policies efficiently.

• This work studies sample-efficient learning of complex policies in the context of robot table tennis.
• It incorporates learning into a hierarchical control framework using a model-free strategy layer (which requires complex reasoning about opponents that is difficult to do in a model-based way), model-based prediction of external objects (which are difficult to control directly with analytic control methods, but governed by learnable and relatively simple laws of physics), and analytic controllers for the robot itself.
• Human demonstrations are used to train dynamics models, which together with the analytic controller allow any robot that is physically capable to play table tennis without training episodes.