Social Signals and Multimedia: Past, Present, Future

Hayley Hung, Cathal Gurrin, Martha Larson, Hatice Gunes, Fabien Ringeval, Elisabeth Andre, Louis-Philippe Morency

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

21 Downloads (Pure)


The rising popularity of Artificial Intelligence (AI) has brought considerable public interest as well faster and more direct transfer of research ideas into practice. One of the aspects of AI that still trails behind considerably is the role of machines in interpreting, enhancing, modeling, generating, and influencing social behavior. Such behavior is captured as social signals, usually by sensors recording multiple modalities, making it classic multimedia data. Such behavior can also be generated by an AI system when interacting with humans. Using AI techniques in combination with multimedia data can be used to pursue multiple goals, two of which are high-lighted here. First, supporting people during social interactions and helping them to fulfil their social needs either actively or passively.Second, improving our understanding of how people collaborate, build relationships, and process self identity. Despite the rise of fields such as Social Signal Processing, a similar panel organised at ACM Multimedia 2014, and an area on social and emotional signal sat the ACM MM since 2014, we argue that we have yet to truly fulfil the potential of the combining social signals and multimedia. This panel asks where we have come far enough and what remaining challenges there are in light of recent global events.

Original languageEnglish
Title of host publicationMM 2021
Subtitle of host publicationProceedings of the 29th ACM International Conference on Multimedia
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Number of pages3
ISBN (Print)978-1-4503-8651-7
Publication statusPublished - 2021
Event29th ACM International Conference on Multimedia, MM 2021 - Virtual, Online, China
Duration: 20 Oct 202124 Oct 2021


Conference29th ACM International Conference on Multimedia, MM 2021
CityVirtual, Online

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • artificial intelligence
  • human social behavior
  • multi-modal machine learning
  • multimedia
  • social signal processing


Dive into the research topics of 'Social Signals and Multimedia: Past, Present, Future'. Together they form a unique fingerprint.

Cite this