On Social Involvement in Mingling Scenarios: Detecting Associates of F-formations in Still Images

Lu Zhang, Hayley Hung

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)
37 Downloads (Pure)


In this paper, we carry out an extensive study of social involvement in free standing conversing groups (the so-called F-formations) from static images. By introducing a novel feature representation, we show that the standard features which have been used to represent full membership in an F-formation cannot be applied to the detection of associates of F-formations due their sparser occurence. We enrich state-of-the-art F-formation modelling by learning a frustum of attention that accounts for the spatial context. That is, F-formation configurations vary with respect to the arrangement of furniture and the non-uniform crowdednessin the space during mingling scenarios. Moroever, the majority of prior works have considered the labelling of conversing groups as an objective task, requiring only a single annotator. However, we show that by embracing the subjectivity of social involvement, we not only generate a richer model of the social interactions in a scene but can use the detected associates to improve initial estimates of the full members of an F-formation. We carry out extensive experimental validation of our proposed approach by collecting a novel set of multi-annotator labels of involvement on two publicly available datasets; The Idiap Poster Data and SALSA data set.

Original languageEnglish
Pages (from-to)1-13
Number of pages13
JournalIEEE Transactions on Affective Computing
Publication statusE-pub ahead of print - 2020


  • F-formations Detection
  • Feature extraction
  • Human Behaviour Analysis
  • Labeling
  • Psychology
  • Semantics
  • Social Group Detection
  • Surveillance
  • Task analysis
  • Visualization

Fingerprint Dive into the research topics of 'On Social Involvement in Mingling Scenarios: Detecting Associates of F-formations in Still Images'. Together they form a unique fingerprint.

Cite this