Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?

Understand

The use of deep reinforcement learning allows for high-dimensional state descriptors, but little is known about how the choice of action representation impacts the learning difficulty and the resulting performance.

• We compare the impact of four different action parameterizations (torques, muscle-activations, target joint angles, and target joint-angle velocities) in terms of learning time, policy robustness, motion quality, and policy query rates.
• Our results are evaluated on a gait-cycle imitation task for multiple planar articulated figures and multiple gaits.
• We demonstrate that the local feedback provided by higher-level action parameterizations can significantly impact the learning, robustness, and quality of the resulting policies.