TY - JOUR
T1 - From Video to Hybrid Simulator
T2 - Exploring Affective Responses toward Non-Verbal Pedestrian Crossing Actions Using Camera and Physiological Sensors
AU - Rao, Shruti
AU - Ghosh, Surjya
AU - Rodriguez, Gerard Pons
AU - Röggla, Thomas
AU - Cesar, Pablo
AU - El Ali, Abdallah
PY - 2023
Y1 - 2023
N2 - Capturing drivers’ affective responses given driving context and driver-pedestrian interactions remains a challenge for designing in-vehicle, empathic interfaces. To address this, we conducted two lab-based studies using camera and physiological sensors. Our first study collected participants’ (N = 21) emotion self-reports and physiological signals (including facial temperatures) toward non-verbal, pedestrian crossing videos from the Joint Attention for Autonomous Driving dataset. Our second study increased realism by employing a hybrid driving simulator setup to capture participants’ affective responses (N = 24) toward enacted, non-verbal pedestrian crossing actions. Key findings showed: (a) non-positive actions in videos elicited higher arousal ratings, whereas different in-video pedestrian crossing actions significantly influenced participants’ physiological signals. (b) Non-verbal pedestrian interactions in the hybrid simulator setup significantly influenced participants’ facial expressions, but not their physiological signals. We contribute to the development of in-vehicle empathic interfaces that draw on behavioral and physiological sensing to in-situ infer driver affective responses during non-verbal pedestrian interactions.
AB - Capturing drivers’ affective responses given driving context and driver-pedestrian interactions remains a challenge for designing in-vehicle, empathic interfaces. To address this, we conducted two lab-based studies using camera and physiological sensors. Our first study collected participants’ (N = 21) emotion self-reports and physiological signals (including facial temperatures) toward non-verbal, pedestrian crossing videos from the Joint Attention for Autonomous Driving dataset. Our second study increased realism by employing a hybrid driving simulator setup to capture participants’ affective responses (N = 24) toward enacted, non-verbal pedestrian crossing actions. Key findings showed: (a) non-positive actions in videos elicited higher arousal ratings, whereas different in-video pedestrian crossing actions significantly influenced participants’ physiological signals. (b) Non-verbal pedestrian interactions in the hybrid simulator setup significantly influenced participants’ facial expressions, but not their physiological signals. We contribute to the development of in-vehicle empathic interfaces that draw on behavioral and physiological sensing to in-situ infer driver affective responses during non-verbal pedestrian interactions.
KW - driver emotion recognition
KW - driving simulator
KW - Empathic car
KW - pedestrian behavior
KW - physiological sensing
KW - thermal sensing
UR - http://www.scopus.com/inward/record.url?scp=85164506453&partnerID=8YFLogxK
U2 - 10.1080/10447318.2023.2224955
DO - 10.1080/10447318.2023.2224955
M3 - Article
AN - SCOPUS:85164506453
SN - 1044-7318
VL - 39
SP - 3213
EP - 3236
JO - International Journal of Human-Computer Interaction
JF - International Journal of Human-Computer Interaction
IS - 16
ER -