Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics

Understand

Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs.

• However, when defining the corresponding optimal control or reinforcement learning problem, it is commonly approximated with fully continuous or fully discrete action spaces.
• These simplifications aim at tailoring the problem to a particular algorithm or solver which may only support one type of action space.
• Alternatively, expert heuristics are used to remove discrete actions from an otherwise continuous space.