Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System

Xingran Ruan; Charaka Palansuriya; Aurora Constantin; Konstantinos Tsiakas

doi:10.1145/3585088.3593882

Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System

Xingran Ruan^*, Charaka Palansuriya, Aurora Constantin, Konstantinos Tsiakas

^*Corresponding author for this work

Human Technology Relations

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

1 Citation (Scopus)

36 Downloads (Pure)

Abstract

The present study aims to investigate the relationship between emotions experienced during learning and metacognition in typically developing (TD) children and those with autism spectrum disorder (ASD). This will assist us in using machine learning (ML) to develop a facial emotion recognition (FER) based intelligent tutor system (ITS) to support children’s metacognitive monitoring process in order to enhance their learning outcomes. In this paper, we first report the results of our preliminary research, which utilized an ML-based FER algorithm to detect four spontaneous epistemic emotions (i.e., neutral, confused, frustrated, and boredom) and six spontaneous basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise). Subsequently, we adapted an application (‘BrainHood’) to create the ‘Meta-BrainHood’, that embedded our proposed ML-based FER algorithm to examine the relationship between facial emotion expressions and metacognitive monitoring performance in TD children and those with ASD. Finally, we outline the future steps in our research, which adopts the outcomes of the first two steps to construct an ITS to improve children’s metacognitive monitoring performance and learning outcomes.

Original language	English
Title of host publication	Proceedings of IDC 2023 - 22nd Annual ACM Interaction Design and Children Conference
Subtitle of host publication	Rediscovering Childhood
Place of Publication	New York
Publisher	ACM
Pages	502-506
Number of pages	5
ISBN (Print)	979-8-4007-0131-3
DOIs	https://doi.org/10.1145/3585088.3593882
Publication status	Published - 2023
Event	22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood, IDC 2023 - Chicago, United States Duration: 19 Jun 2023 → 22 Jun 2023

Conference

Conference	22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood, IDC 2023
Country/Territory	United States
City	Chicago
Period	19/06/23 → 22/06/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

facial emotion recognition
Intelligent tutor system
learning outcomes
metacognitive monitoring process

Access to Document

10.1145/3585088.3593882

3585088.3593882Final published version, 1.15 MB

Cite this

@inproceedings{4186ffbd31514cd98ff76f05728a0eee,

title = "Supporting Children{\textquoteright}s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System",

abstract = "The present study aims to investigate the relationship between emotions experienced during learning and metacognition in typically developing (TD) children and those with autism spectrum disorder (ASD). This will assist us in using machine learning (ML) to develop a facial emotion recognition (FER) based intelligent tutor system (ITS) to support children{\textquoteright}s metacognitive monitoring process in order to enhance their learning outcomes. In this paper, we first report the results of our preliminary research, which utilized an ML-based FER algorithm to detect four spontaneous epistemic emotions (i.e., neutral, confused, frustrated, and boredom) and six spontaneous basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise). Subsequently, we adapted an application ({\textquoteleft}BrainHood{\textquoteright}) to create the {\textquoteleft}Meta-BrainHood{\textquoteright}, that embedded our proposed ML-based FER algorithm to examine the relationship between facial emotion expressions and metacognitive monitoring performance in TD children and those with ASD. Finally, we outline the future steps in our research, which adopts the outcomes of the first two steps to construct an ITS to improve children{\textquoteright}s metacognitive monitoring performance and learning outcomes.",

keywords = "facial emotion recognition, Intelligent tutor system, learning outcomes, metacognitive monitoring process",

author = "Xingran Ruan and Charaka Palansuriya and Aurora Constantin and Konstantinos Tsiakas",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood, IDC 2023 ; Conference date: 19-06-2023 Through 22-06-2023",

year = "2023",

doi = "10.1145/3585088.3593882",

language = "English",

isbn = "979-8-4007-0131-3",

pages = "502--506",

booktitle = "Proceedings of IDC 2023 - 22nd Annual ACM Interaction Design and Children Conference",

publisher = "ACM",

address = "United States",

}

Ruan, X, Palansuriya, C, Constantin, A & Tsiakas, K 2023, Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System. in Proceedings of IDC 2023 - 22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood. ACM, New York, pp. 502-506, 22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood, IDC 2023, Chicago, United States, 19/06/23. https://doi.org/10.1145/3585088.3593882

Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System. / Ruan, Xingran; Palansuriya, Charaka; Constantin, Aurora et al.
Proceedings of IDC 2023 - 22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood. New York: ACM, 2023. p. 502-506.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System

AU - Ruan, Xingran

AU - Palansuriya, Charaka

AU - Constantin, Aurora

AU - Tsiakas, Konstantinos

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - The present study aims to investigate the relationship between emotions experienced during learning and metacognition in typically developing (TD) children and those with autism spectrum disorder (ASD). This will assist us in using machine learning (ML) to develop a facial emotion recognition (FER) based intelligent tutor system (ITS) to support children’s metacognitive monitoring process in order to enhance their learning outcomes. In this paper, we first report the results of our preliminary research, which utilized an ML-based FER algorithm to detect four spontaneous epistemic emotions (i.e., neutral, confused, frustrated, and boredom) and six spontaneous basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise). Subsequently, we adapted an application (‘BrainHood’) to create the ‘Meta-BrainHood’, that embedded our proposed ML-based FER algorithm to examine the relationship between facial emotion expressions and metacognitive monitoring performance in TD children and those with ASD. Finally, we outline the future steps in our research, which adopts the outcomes of the first two steps to construct an ITS to improve children’s metacognitive monitoring performance and learning outcomes.

AB - The present study aims to investigate the relationship between emotions experienced during learning and metacognition in typically developing (TD) children and those with autism spectrum disorder (ASD). This will assist us in using machine learning (ML) to develop a facial emotion recognition (FER) based intelligent tutor system (ITS) to support children’s metacognitive monitoring process in order to enhance their learning outcomes. In this paper, we first report the results of our preliminary research, which utilized an ML-based FER algorithm to detect four spontaneous epistemic emotions (i.e., neutral, confused, frustrated, and boredom) and six spontaneous basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise). Subsequently, we adapted an application (‘BrainHood’) to create the ‘Meta-BrainHood’, that embedded our proposed ML-based FER algorithm to examine the relationship between facial emotion expressions and metacognitive monitoring performance in TD children and those with ASD. Finally, we outline the future steps in our research, which adopts the outcomes of the first two steps to construct an ITS to improve children’s metacognitive monitoring performance and learning outcomes.

KW - facial emotion recognition

KW - Intelligent tutor system

KW - learning outcomes

KW - metacognitive monitoring process

UR - http://www.scopus.com/inward/record.url?scp=85169089546&partnerID=8YFLogxK

U2 - 10.1145/3585088.3593882

DO - 10.1145/3585088.3593882

M3 - Conference contribution

AN - SCOPUS:85169089546

SN - 979-8-4007-0131-3

SP - 502

EP - 506

BT - Proceedings of IDC 2023 - 22nd Annual ACM Interaction Design and Children Conference

PB - ACM

CY - New York

T2 - 22nd Annual ACM Interaction Design and Children Conference: Rediscovering Childhood, IDC 2023

Y2 - 19 June 2023 through 22 June 2023

ER -

Supporting Children’s Metacognition with a Facial Emotion Recognition based Intelligent Tutor System

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this