Real-time multimodal feedback with the CPR tutor

Daniele Di Mitri; Jan Schneider; Kevin Trebing; Sasa Sopka; Marcus Specht; Hendrik Drachsler

doi:10.1007/978-3-030-52237-7_12

Real-time multimodal feedback with the CPR tutor

Daniele Di Mitri, Jan Schneider, Kevin Trebing, Sasa Sopka, Marcus Specht, Hendrik Drachsler

Web Information Systems

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

40 Citations (Scopus)

49 Downloads (Pure)

Abstract

We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

Original language	English
Title of host publication	Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I
Subtitle of host publication	21st International Conference, AIED 2020, Proceedings, Part I
Editors	Ig Ibert Bittencourt, Mutlu Cukurova, Rose Luckin, Kasia Muldner, Eva Millán
Place of Publication	Cham
Publisher	Springer
Pages	141-152
Number of pages	12
ISBN (Electronic)	978-3-030-52237-7
ISBN (Print)	978-3-030-52236-0
DOIs	https://doi.org/10.1007/978-3-030-52237-7_12
Publication status	Published - 2020
Event	21st International Conference on Artificial Intelligence in Education, AIED 2020 - Ifrane, Morocco Duration: 6 Jul 2020 → 10 Jul 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12163 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	21st International Conference on Artificial Intelligence in Education, AIED 2020
Country/Territory	Morocco
City	Ifrane
Period	6/07/20 → 10/07/20

Bibliographical note

Virtual/online event due to COVID-19
Accepted author manuscript

Access to Document

10.1007/978-3-030-52237-7_12

Real_time_Multimodal_Feedback_with_the_CPR_Tutor[2]Accepted author manuscript, 3.57 MB

Cite this

Di Mitri, D., Schneider, J., Trebing, K., Sopka, S., Specht, M., & Drachsler, H. (2020). Real-time multimodal feedback with the CPR tutor. In I. I. Bittencourt, M. Cukurova, R. Luckin, K. Muldner, & E. Millán (Eds.), Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I: 21st International Conference, AIED 2020, Proceedings, Part I (pp. 141-152). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12163 LNAI). Springer. https://doi.org/10.1007/978-3-030-52237-7_12

Di Mitri, Daniele ; Schneider, Jan ; Trebing, Kevin et al. / Real-time multimodal feedback with the CPR tutor. Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I: 21st International Conference, AIED 2020, Proceedings, Part I. editor / Ig Ibert Bittencourt ; Mutlu Cukurova ; Rose Luckin ; Kasia Muldner ; Eva Millán. Cham : Springer, 2020. pp. 141-152 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{2683220fbae54ffd8a54906f8b1bb3b5,

title = "Real-time multimodal feedback with the CPR tutor",

abstract = "We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.",

author = "{Di Mitri}, Daniele and Jan Schneider and Kevin Trebing and Sasa Sopka and Marcus Specht and Hendrik Drachsler",

note = "Virtual/online event due to COVID-19 Accepted author manuscript; 21st International Conference on Artificial Intelligence in Education, AIED 2020 ; Conference date: 06-07-2020 Through 10-07-2020",

year = "2020",

doi = "10.1007/978-3-030-52237-7_12",

language = "English",

isbn = "978-3-030-52236-0",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "141--152",

editor = "Bittencourt, {Ig Ibert} and Mutlu Cukurova and Rose Luckin and Kasia Muldner and Eva Mill{\'a}n",

booktitle = "Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I",

}

Di Mitri, D, Schneider, J, Trebing, K, Sopka, S, Specht, M & Drachsler, H 2020, Real-time multimodal feedback with the CPR tutor. in II Bittencourt, M Cukurova, R Luckin, K Muldner & E Millán (eds), Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I: 21st International Conference, AIED 2020, Proceedings, Part I. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12163 LNAI, Springer, Cham, pp. 141-152, 21st International Conference on Artificial Intelligence in Education, AIED 2020, Ifrane, Morocco, 6/07/20. https://doi.org/10.1007/978-3-030-52237-7_12

Real-time multimodal feedback with the CPR tutor. / Di Mitri, Daniele; Schneider, Jan; Trebing, Kevin et al.
Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I: 21st International Conference, AIED 2020, Proceedings, Part I. ed. / Ig Ibert Bittencourt; Mutlu Cukurova; Rose Luckin; Kasia Muldner; Eva Millán. Cham: Springer, 2020. p. 141-152 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12163 LNAI).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Real-time multimodal feedback with the CPR tutor

AU - Di Mitri, Daniele

AU - Schneider, Jan

AU - Trebing, Kevin

AU - Sopka, Sasa

AU - Specht, Marcus

AU - Drachsler, Hendrik

N1 - Virtual/online event due to COVID-19 Accepted author manuscript

PY - 2020

Y1 - 2020

N2 - We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

AB - We developed the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects mistakes using recurrent neural networks for real-time time-series classification. From a multimodal data stream consisting of kinematic and electromyographic data, the CPR Tutor system automatically detects the chest compressions, which are then classified and assessed according to five performance indicators. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. To test the validity of the CPR Tutor, we first collected the data corpus from 10 experts used for model training. Hence, to test the impact of the feedback functionality, we ran a user study involving 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: 1) an architecture design, 2) a methodological approach to design multimodal feedback and 3) a field study on real-time feedback for CPR training.

UR - http://www.scopus.com/inward/record.url?scp=85089609447&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-52237-7_12

DO - 10.1007/978-3-030-52237-7_12

M3 - Conference contribution

SN - 978-3-030-52236-0

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 141

EP - 152

BT - Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I

A2 - Bittencourt, Ig Ibert

A2 - Cukurova, Mutlu

A2 - Luckin, Rose

A2 - Muldner, Kasia

A2 - Millán, Eva

PB - Springer

CY - Cham

T2 - 21st International Conference on Artificial Intelligence in Education, AIED 2020

Y2 - 6 July 2020 through 10 July 2020

ER -

Di Mitri D, Schneider J, Trebing K, Sopka S, Specht M, Drachsler H. Real-time multimodal feedback with the CPR tutor. In Bittencourt II, Cukurova M, Luckin R, Muldner K, Millán E, editors, Artificial Intelligence in Education- 21st International Conference, AIED 2020, Proceedings, Part I: 21st International Conference, AIED 2020, Proceedings, Part I. Cham: Springer. 2020. p. 141-152. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-52237-7_12

Real-time multimodal feedback with the CPR tutor

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this