Multimodal markers for technology-independent integration of augmented reality devices and surgical navigation systems

Mohamed Benmahdjoub*, Wiro J. Niessen, Eppo B. Wolvius, Theo van Walsum

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

4 Citations (Scopus)
69 Downloads (Pure)

Abstract

Augmented reality (AR) permits the visualization of pre-operative data in the surgical field of view of the surgeon. This requires the alignment of the AR device’s coordinate system with the used navigation/tracking system. We propose a multimodal marker approach to align an AR device with a tracking system: in our implementation, an electromagnetic tracking system (EMTS). The solution makes use of a calibration method which determines the relationship between a 2D pattern detected by an RGB camera and an electromagnetic sensor of the EMTS. This allowed the projection of a 3D skull model on its physical counterpart. This projection was evaluated using a monocular camera and an optical see-through device (HoloLens 2) (https://www.microsoft.com/en-us/hololens/) achieving an accuracy of less than 2.5 mm in the image plane of the HoloLens 2 (HL2). Additionally, 10 volunteers participated in a user study consisting of an alignment task of a pointer with 25 projections viewed through the HL2. The participants achieved a mean error of 2.7 1.3 mm and 2.9 2.9 in positional and orientation error. This study showcases the feasibility of the approach, provides an evaluation of the alignment, and finally, discusses its advantages and limitations.

Original languageEnglish
Pages (from-to)1637-1650
Number of pages14
JournalVirtual Reality
Volume26
Issue number4
DOIs
Publication statusPublished - 2022

Keywords

  • Alignment
  • Augmented reality
  • Calibration
  • Feasibility studies
  • Mixed reality
  • Navigation system
  • Surgical interventions

Fingerprint

Dive into the research topics of 'Multimodal markers for technology-independent integration of augmented reality devices and surgical navigation systems'. Together they form a unique fingerprint.

Cite this