Flycon: Real-time environment-independent multi-view human pose estimation with aerial vehicles

Tobias Nageli, Samuel Oberholzer, Silvan Pluss, Javier Alonso-Mora, Otmar Hilliges

Research output: Contribution to journalArticleScientificpeer-review

6 Citations (Scopus)
7 Downloads (Pure)


We propose a real-time method for the infrastructure-free estimation of articulated human motion. The approach leverages a swarm of camer aequipped flying robots and jointly optimizes the swarm's and skeletal states, which include the 3D joint positions and a set of bones. Our method allows to track the motion of human subjects, for example an athlete, over long time horizons and long distances, in challenging settings and at large scale, where fixed infrastructure approaches are not applicable. The proposed algorithm uses active infra-red markers, runs in real-time and accurately estimates robot and human pose parameters online without the need for accurately calibrated or stationary mounted cameras. Our method i) estimates a global coordinate frame for the MAV swarm, ii) jointly optimizes the human pose and relative camera positions, and iii) estimates the length of the human bones. The entire swarm is then controlled via a model predictive controller to maximize visibility of the subject from multiple viewpoints even under fast motion such as jumping or jogging. We demonstrate our method in a number of difficult scenarios including capture of long locomotion sequences at the scale of a triplex gym, in non-planar terrain, while climbing and in outdoor scenarios. CCS Concepts: • Computing methodologies → Motion capture; Reconstruction; Robotic planning; Motion path planning;.

Original languageEnglish
Article number182
Number of pages14
JournalACM Transactions on Graphics
Issue number6
Publication statusPublished - 2018

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Human pose estimation
  • Robotics


Dive into the research topics of 'Flycon: Real-time environment-independent multi-view human pose estimation with aerial vehicles'. Together they form a unique fingerprint.

Cite this