Reconstructing occluded Elevation Information in Terrain Maps with Self-supervised Learning

Maximilian Stolzle, Takahiro Miki, Levin Gerdes, Martin Azkarate, Marco Hutter

Research output: Contribution to journalArticleScientificpeer-review

3 Citations (Scopus)
45 Downloads (Pure)


Accurate and complete terrain maps enhance the awareness of autonomous robots and enable safe and optimal path planning. Rocks and topography often create occlusions and lead to missing elevation information in the Digital Elevation Map (DEM). Currently, these occluded areas are either fully avoided during motion planning or the missing values in the elevation map are filled-in using traditional interpolation, diffusion or patch-matching techniques. These methods cannot leverage the high-level terrain characteristics and the geometric constraints of line of sight we humans use intuitively to predict occluded areas. We introduce a self-supervised learning approach capable of training on real-world data without a need for ground-truth information to reconstruct the occluded areas in the DEMs. We accomplish this by adding artificial occlusion to the incomplete elevation maps constructed on a real robot by performing ray casting. We first evaluate a supervised learning approach on synthetic data for which we have the full ground-truth available and subsequently move to several real-world datasets. These real-world datasets were recorded during exploration of both structured and unstructured terrain with a legged robot, and additionally in a planetary scenario on Lunar analogue terrain. We state a significant improvement compared to the baseline methods both on synthetic terrain and for the real-world datasets. Our neural network is able to run in real-time on both CPU and GPU with suitable sampling rates for autonomous ground robots. We motivate the applicability of reconstructing occlusion in elevation maps with preliminary motion planning experiments.

Original languageEnglish
Pages (from-to)1697-1704
JournalIEEE Robotics and Automation Letters
Issue number2
Publication statusPublished - 2022

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • AI-enabled robotics
  • Casting
  • Image reconstruction
  • Mapping
  • Neural networks
  • Planning
  • Robots
  • Sensors
  • Training


Dive into the research topics of 'Reconstructing occluded Elevation Information in Terrain Maps with Self-supervised Learning'. Together they form a unique fingerprint.

Cite this