Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics

Research output: Contribution to journalArticleScientificpeer-review

5 Citations (Scopus)
78 Downloads (Pure)


Human head orientation estimation has been of interest because head orientation serves as a cue to directed social attention. Most existing approaches rely on visual and high-fidelity sensor inputs and deep learning strategies that do not consider the social context of unstructured and crowded mingling scenarios. We show that alternative inputs, like speaking status, body location, orientation, and acceleration contribute towards head orientation estimation. These are especially useful in crowded and in-the-wild settings where visual features are either uninformative due to occlusions or prohibitive to acquire due to physical space limitations and concerns of ecological validity. We argue that head orientation estimation in such social settings needs to account for the physically evolving interaction space formed by all the individuals in the group. To this end, we propose an LSTM-based head orientation estimation method that combines the hidden representations of the group members. Our framework jointly predicts head orientations of all group members and is applicable to groups of different sizes. We explain the contribution of different modalities to model performance in head orientation estimation. The proposed model outperforms baseline methods that do not explicitly consider the group context, and generalizes to an unseen dataset from a different social event.

Original languageEnglish
Article number35
Number of pages22
JournalProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Issue number1
Publication statusPublished - 2021


  • head orientation estimation
  • interaction dynamics
  • scene understanding


Dive into the research topics of 'Multimodal Joint Head Orientation Estimation in Interacting Groups via Proxemics and Interaction Dynamics'. Together they form a unique fingerprint.

Cite this