Structured Evolution with Compact Architectures for Scalable Policy Optimization

Understand

We present a new method of blackbox optimization via gradient approximation with the use of structured random orthogonal matrices, providing more accurate estimators than baselines and with provable theoretical guarantees.

• We show that this algorithm can be successfully applied to learn better quality compact policies than those using standard gradient estimation techniques.
• The compact policies we learn have several advantages over unstructured ones, including faster training algorithms and faster inference.
• These benefits are important when the policy is deployed on real hardware with limited resources.