Parallel Real-Time Tracking and 3D Reconstruction with TBB for Intelligent Control and Smart Sensing Framework

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

46 Downloads (Pure)


Recent advancements in aircraft controllers and increasingly flexible aircraft designs create the need for adaptive and intelligent control systems. To correctly capture the motion of a flexible aircraft wing and provide feedback to the controller large number of states (nodes along the span) must be monitored. Visual sensing method can provide the flexibility that is needed for this type of smart sensing and control. Visual sensing, however, requires to capture and track keypoint features (marker tracking), and detecting these keypoints from a feature-rich image can be a computationally intensive task. The computational effort greatly increases with larger images and or when an image stereo pair is used to find matching keypoints. In this study, a parallel approach is presented with Threading Building Blocks (TBB) to extract corresponding keypoints from an image-stereo pair and apply triangulation with the DLT (Direct Linear Transform) method to reconstruct the 3D position of the object in space. Furthermore, a flexible simulation framework is set-up using for smart sensing with a coupled unsteady aeroservoelastic model of a 3D wing and a visual model to test the method for intelligent control feedback in a simulation environment. The methodology is tested in a laboratory environment with a stereo camera setup and virtual environment, where the virtual camera parameters are reconstructed to meet a stereo setup. The approach used in this study aims to advance the are of smart sensing, particularly in the context of real-time state estimation of aeroelastic structures and control feedback. The parallel approach shows a great improvement of speed in the order of 3-6 depending on the image size, allowing real- time computation from live image stream at 50 fps. A comparison is made of parallel versus non-parallel approach for various image sizes.
Original languageEnglish
Title of host publicationAIAA Scitech 2020 Forum
Subtitle of host publication6-10 January 2020, Orlando, FL
PublisherAmerican Institute of Aeronautics and Astronautics Inc. (AIAA)
Number of pages23
ISBN (Electronic)978-1-62410-595-1
Publication statusPublished - 2020
EventAIAA Scitech 2020 Forum - Orlando, United States
Duration: 6 Jan 202010 Jan 2020


ConferenceAIAA Scitech 2020 Forum
CountryUnited States

Fingerprint Dive into the research topics of 'Parallel Real-Time Tracking and 3D Reconstruction with TBB for Intelligent Control and Smart Sensing Framework'. Together they form a unique fingerprint.

  • Cite this

    Mkhoyan, T., de Visser, C., & De Breuker, R. (2020). Parallel Real-Time Tracking and 3D Reconstruction with TBB for Intelligent Control and Smart Sensing Framework. In AIAA Scitech 2020 Forum: 6-10 January 2020, Orlando, FL [AIAA 2020-2252] American Institute of Aeronautics and Astronautics Inc. (AIAA).