Exploring Personal Memories and Video Content as Context for Facial Behavior in Predictions of Video-Induced Emotions

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

37 Downloads (Pure)

Abstract

Empirical evidence suggests that the emotional meaning of facial behavior in isolation is often ambiguous in real-world conditions. While humans complement interpretations of others' faces with additional reasoning about context, automated approaches rarely display such context-sensitivity. Empirical findings indicate that the personal memories triggered by videos are crucial for predicting viewers' emotional response to such videos ?- in some cases, even more so than the video's audiovisual content. In this article, we explore the benefits of personal memories as context for facial behavior analysis. We conduct a series of multimodal machine learning experiments combining the automatic analysis of video-viewers' faces with that of two types of context information for affective predictions: \beginenumerate∗[label=(\arabic∗)] \item self-reported free-text descriptions of triggered memories and \item a video's audiovisual content \endenumerate∗. Our results demonstrate that both sources of context provide models with information about variation in viewers' affective responses that complement facial analysis and each other.

Original languageEnglish
Title of host publicationICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Pages153-162
Number of pages10
ISBN (Print)978-1-4503-7581-8
DOIs
Publication statusPublished - 2020
Event22nd ACM International Conference on Multimodal Interaction, ICMI 2020 - Virtual, Online, Netherlands
Duration: 25 Oct 202029 Oct 2020
Conference number: 22

Conference

Conference22nd ACM International Conference on Multimodal Interaction, ICMI 2020
Abbreviated titleICMI 2020
CountryNetherlands
Period25/10/2029/10/20

Keywords

  • affect detection
  • context-awareness
  • emotion recognition

Fingerprint

Dive into the research topics of 'Exploring Personal Memories and Video Content as Context for Facial Behavior in Predictions of Video-Induced Emotions'. Together they form a unique fingerprint.

Cite this