Persistent self-supervised learning: From stereo to monocular vision for obstacle avoidance

Kevin van Hecke*, Guido de Croon, Laurens van der Maaten, Daniel Hennes, Dario Izzo

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

14 Citations (Scopus)
162 Downloads (Pure)


Self-supervised learning is a reliable learning mechanism in which a robot uses an original, trusted sensor cue for training to recognize an additional, complementary sensor cue. We study for the first time in self-supervised learning how a robot’s learning behavior should be organized, so that the robot can keep performing its task in the case that the original cue becomes unavailable. We study this persistent form of self-supervised learning in the context of a flying robot that has to avoid obstacles based on distance estimates from the visual cue of stereo vision. Over time it will learn to also estimate distances based on monocular appearance cues. A strategy is introduced that has the robot switch from flight based on stereo to flight based on monocular vision, with stereo vision purely used as “training wheels” to avoid imminent collisions. This strategy is shown to be an effective approach to the “feedback-induced data bias” problem as also experienced in learning from demonstration. Both simulations and real-world experiments with a stereo vision equipped ARDrone2 show the feasibility of this approach, with the robot successfully using monocular vision to avoid obstacles in a 5 × 5 m room. The experiments show the potential of persistent self-supervised learning as a robust learning approach to enhance the capabilities of robots. Moreover, the abundant training data coming from the own sensors allow to gather large data sets necessary for deep learning approaches.

Original languageEnglish
Pages (from-to)186-206
Number of pages21
JournalInternational Journal of Micro Air Vehicles
Issue number2
Publication statusPublished - 2018


  • monocular depth estimation
  • Persistent self-supervised learning
  • robotics
  • stereo vision


Dive into the research topics of 'Persistent self-supervised learning: From stereo to monocular vision for obstacle avoidance'. Together they form a unique fingerprint.

Cite this