From Video to Hybrid Simulator: Exploring Affective Responses toward Non-Verbal Pedestrian Crossing Actions Using Camera and Physiological Sensors

Shruti Rao, Surjya Ghosh, Gerard Pons Rodriguez, Thomas Röggla, Pablo Cesar, Abdallah El Ali*

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

2 Citations (Scopus)
27 Downloads (Pure)

Abstract

Capturing drivers’ affective responses given driving context and driver-pedestrian interactions remains a challenge for designing in-vehicle, empathic interfaces. To address this, we conducted two lab-based studies using camera and physiological sensors. Our first study collected participants’ (N = 21) emotion self-reports and physiological signals (including facial temperatures) toward non-verbal, pedestrian crossing videos from the Joint Attention for Autonomous Driving dataset. Our second study increased realism by employing a hybrid driving simulator setup to capture participants’ affective responses (N = 24) toward enacted, non-verbal pedestrian crossing actions. Key findings showed: (a) non-positive actions in videos elicited higher arousal ratings, whereas different in-video pedestrian crossing actions significantly influenced participants’ physiological signals. (b) Non-verbal pedestrian interactions in the hybrid simulator setup significantly influenced participants’ facial expressions, but not their physiological signals. We contribute to the development of in-vehicle empathic interfaces that draw on behavioral and physiological sensing to in-situ infer driver affective responses during non-verbal pedestrian interactions.

Original languageEnglish
Pages (from-to)3213-3236
Number of pages24
JournalInternational Journal of Human-Computer Interaction
Volume39
Issue number16
DOIs
Publication statusPublished - 2023

Keywords

  • driver emotion recognition
  • driving simulator
  • Empathic car
  • pedestrian behavior
  • physiological sensing
  • thermal sensing

Fingerprint

Dive into the research topics of 'From Video to Hybrid Simulator: Exploring Affective Responses toward Non-Verbal Pedestrian Crossing Actions Using Camera and Physiological Sensors'. Together they form a unique fingerprint.

Cite this