End-to-end neural network based optimal quadcopter control

Robin Ferede*, Guido de Croon, Christophe De Wagter, Dario Izzo

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

17 Downloads (Pure)


Developing optimal controllers for aggressive high-speed quadcopter flight poses significant challenges in robotics. Recent trends in the field involve utilizing neural network controllers trained through supervised or reinforcement learning. However, the sim-to-real transfer introduces a reality gap, requiring the use of robust inner loop controllers during real flights, which limits the network's control authority and flight performance. In this paper, we investigate for the first time, an end-to-end neural network controller, addressing the reality gap issue without being restricted by an inner-loop controller. The networks, referred to as G&CNets, are trained to learn an energy-optimal policy mapping the quadcopter's state to rpm commands using an optimal trajectory dataset. In hover-to-hover flights, we identified the unmodeled moments as a significant contributor to the reality gap. To mitigate this, we propose an adaptive control strategy that works by learning from optimal trajectories of a system affected by constant external pitch, roll and yaw moments. In real test flights, this model mismatch is estimated onboard and fed to the network to obtain the optimal rpm command. We demonstrate the effectiveness of our method by performing energy-optimal hover-to-hover flights with and without moment feedback. Finally, we compare the adaptive controller to a state-of-the-art differential-flatness-based controller in a consecutive waypoint flight and demonstrate the advantages of our method in terms of energy optimality and robustness.

Original languageEnglish
Article number104588
Number of pages11
JournalRobotics and Autonomous Systems
Publication statusPublished - 2024


This work was supported by the European Space Agency.This research was co-funded under the Discovery programme of, and funded by, the European Space Agency.


  • End-to-end control
  • G&CNet
  • Optimal control
  • Reality gap
  • Sim-to-real transfer
  • Supervised learning


Dive into the research topics of 'End-to-end neural network based optimal quadcopter control'. Together they form a unique fingerprint.

Cite this