Real-time multimodal feedback with the CPR tutor

Daniele Di Mitri, Jan Schneider, Kevin Trebing, Sasa Sopka, Marcus Specht, Hendrik Drachsler

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

40 Citations (Scopus)
52 Downloads (Pure)


We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

Original languageEnglish
Title of host publicationArtificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I
Subtitle of host publication21st International Conference, AIED 2020, Proceedings, Part I
EditorsIg Ibert Bittencourt, Mutlu Cukurova, Rose Luckin, Kasia Muldner, Eva Millán
Place of PublicationCham
Number of pages12
ISBN (Electronic)978-3-030-52237-7
ISBN (Print)978-3-030-52236-0
Publication statusPublished - 2020
Event21st International Conference on Artificial Intelligence in Education, AIED 2020 - Ifrane, Morocco
Duration: 6 Jul 202010 Jul 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12163 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference21st International Conference on Artificial Intelligence in Education, AIED 2020

Bibliographical note

Virtual/online event due to COVID-19
Accepted author manuscript


Dive into the research topics of 'Real-time multimodal feedback with the CPR tutor'. Together they form a unique fingerprint.

Cite this