Asynchronous Reinforcement Learning for Real-Time Control of Physical Robots

Understand

An oft-ignored challenge of real-world reinforcement learning is that the real world does not pause when agents make learning updates.

• As standard simulated environments do not address this real-time aspect of learning, most available implementations of RL algorithms process environment interactions and learning updates sequentially.
• As a consequence, when such implementations are deployed in the real world, they may make decisions based on significantly delayed observations and not act responsively.
• Asynchronous learning has been proposed to solve this issue, but no systematic comparison between sequential and asynchronous reinforcement learning was conducted using real-world environments.