Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight

Understand

Autonomous drone racing has risen as a challenging robotic benchmark for testing the limits of learning, perception, planning, and control.

• Expert human pilots are able to fly a drone through a race track by mapping pixels from a single camera directly to control commands.
• Recent works in autonomous drone racing attempting direct pixel-to-commands control policies have relied on either intermediate representations that simplify the observation space or performed extensive bootstrapping using Imitation Learning (IL).
• This paper leverages DreamerV3 to train visuomotor policies capable of agile flight through a racetrack using only pixels as observations.