Multimodal Conversational Events Estimation in Complex Social Scenes

Litian Li

doi:10.1145/3716553.3750821

Multimodal Conversational Events Estimation in Complex Social Scenes

^*Corresponding author for this work

Pattern Recognition and Bioinformatics

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

Abstract

Conversational events, such as speaking turns, backchannels, topic changes, and laughter, are central to the structure of multiparty interaction and play a key role in shaping its dynamics. However, detecting such events in real world social settings remains challenging due to perceptual ambiguity, visual occlusion, signal noise, and limitations in acquiring high quality audio data. This work addresses these challenges by focusing on spontaneous interactions in socially complex and privacy sensitive environments, exploring multimodal, nonverbal cues that do not rely on audio. The goal is to develop a novel modeling approach for group context awareness to infer conversational events and support social scene understanding under real world constraints..

Original language	English
Title of host publication	ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction
Editors	Ram Subramanian, Yukiko I. Nakano, Tom Gedeon, Mohan Kankanhalli, Tanaya Guha, Jainendra Shukla, Gelareh Mohammadi, Oya Celiktutan
Place of Publication	New York, NY
Publisher	Association for Computing Machinery (ACM)
Pages	716-720
Number of pages	5
ISBN (Electronic)	9798400714993
DOIs	https://doi.org/10.1145/3716553.3750821
Publication status	Published - 2025
Event	27th International Conference on Multimodal Interaction, ICMI 2025 - Canberra, Australia Duration: 13 Oct 2025 → 17 Oct 2025

Conference

Conference	27th International Conference on Multimodal Interaction, ICMI 2025
Country/Territory	Australia
City	Canberra
Period	13/10/25 → 17/10/25

Bibliographical note

Green Open Access added to TU Delft Institutional Repository as part of the Taverne amendment. More information about this copyright law amendment can be found at https://www.openaccess.nl. Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Human-centered computing
Multimodal interaction
Multimodal Machine Learning
Social Signal Processing

Access to Document

10.1145/3716553.3750821

3716553.3750821Final published version, 991 KB

Cite this

Li, L. (2025). Multimodal Conversational Events Estimation in Complex Social Scenes. In R. Subramanian, Y. I. Nakano, T. Gedeon, M. Kankanhalli, T. Guha, J. Shukla, G. Mohammadi, & O. Celiktutan (Eds.), ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction (pp. 716-720). Association for Computing Machinery (ACM). https://doi.org/10.1145/3716553.3750821

Li, Litian. / Multimodal Conversational Events Estimation in Complex Social Scenes. ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction. editor / Ram Subramanian ; Yukiko I. Nakano ; Tom Gedeon ; Mohan Kankanhalli ; Tanaya Guha ; Jainendra Shukla ; Gelareh Mohammadi ; Oya Celiktutan. New York, NY : Association for Computing Machinery (ACM), 2025. pp. 716-720

@inproceedings{7ed6048f9e3b413296049a0603762b98,

title = "Multimodal Conversational Events Estimation in Complex Social Scenes",

abstract = "Conversational events, such as speaking turns, backchannels, topic changes, and laughter, are central to the structure of multiparty interaction and play a key role in shaping its dynamics. However, detecting such events in real world social settings remains challenging due to perceptual ambiguity, visual occlusion, signal noise, and limitations in acquiring high quality audio data. This work addresses these challenges by focusing on spontaneous interactions in socially complex and privacy sensitive environments, exploring multimodal, nonverbal cues that do not rely on audio. The goal is to develop a novel modeling approach for group context awareness to infer conversational events and support social scene understanding under real world constraints..",

keywords = "Human-centered computing, Multimodal interaction, Multimodal Machine Learning, Social Signal Processing",

author = "Litian Li",

note = "Green Open Access added to TU Delft Institutional Repository as part of the Taverne amendment. More information about this copyright law amendment can be found at https://www.openaccess.nl. Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 27th International Conference on Multimodal Interaction, ICMI 2025 ; Conference date: 13-10-2025 Through 17-10-2025",

year = "2025",

doi = "10.1145/3716553.3750821",

language = "English",

pages = "716--720",

editor = "Ram Subramanian and Nakano, \{Yukiko I.\} and Tom Gedeon and Mohan Kankanhalli and Tanaya Guha and Jainendra Shukla and Gelareh Mohammadi and Oya Celiktutan",

booktitle = "ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

}

Li, L 2025, Multimodal Conversational Events Estimation in Complex Social Scenes. in R Subramanian, YI Nakano, T Gedeon, M Kankanhalli, T Guha, J Shukla, G Mohammadi & O Celiktutan (eds), ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction. Association for Computing Machinery (ACM), New York, NY, pp. 716-720, 27th International Conference on Multimodal Interaction, ICMI 2025, Canberra, Australian Capital Territory, Australia, 13/10/25. https://doi.org/10.1145/3716553.3750821

Multimodal Conversational Events Estimation in Complex Social Scenes. / Li, Litian.
ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction. ed. / Ram Subramanian; Yukiko I. Nakano; Tom Gedeon; Mohan Kankanhalli; Tanaya Guha; Jainendra Shukla; Gelareh Mohammadi; Oya Celiktutan. New York, NY: Association for Computing Machinery (ACM), 2025. p. 716-720.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Multimodal Conversational Events Estimation in Complex Social Scenes

AU - Li, Litian

N1 - Green Open Access added to TU Delft Institutional Repository as part of the Taverne amendment. More information about this copyright law amendment can be found at https://www.openaccess.nl. Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2025

Y1 - 2025

N2 - Conversational events, such as speaking turns, backchannels, topic changes, and laughter, are central to the structure of multiparty interaction and play a key role in shaping its dynamics. However, detecting such events in real world social settings remains challenging due to perceptual ambiguity, visual occlusion, signal noise, and limitations in acquiring high quality audio data. This work addresses these challenges by focusing on spontaneous interactions in socially complex and privacy sensitive environments, exploring multimodal, nonverbal cues that do not rely on audio. The goal is to develop a novel modeling approach for group context awareness to infer conversational events and support social scene understanding under real world constraints..

AB - Conversational events, such as speaking turns, backchannels, topic changes, and laughter, are central to the structure of multiparty interaction and play a key role in shaping its dynamics. However, detecting such events in real world social settings remains challenging due to perceptual ambiguity, visual occlusion, signal noise, and limitations in acquiring high quality audio data. This work addresses these challenges by focusing on spontaneous interactions in socially complex and privacy sensitive environments, exploring multimodal, nonverbal cues that do not rely on audio. The goal is to develop a novel modeling approach for group context awareness to infer conversational events and support social scene understanding under real world constraints..

KW - Human-centered computing

KW - Multimodal interaction

KW - Multimodal Machine Learning

KW - Social Signal Processing

UR - https://www.scopus.com/pages/publications/105022245922

U2 - 10.1145/3716553.3750821

DO - 10.1145/3716553.3750821

M3 - Conference contribution

AN - SCOPUS:105022245922

SP - 716

EP - 720

BT - ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction

A2 - Subramanian, Ram

A2 - Nakano, Yukiko I.

A2 - Gedeon, Tom

A2 - Kankanhalli, Mohan

A2 - Guha, Tanaya

A2 - Shukla, Jainendra

A2 - Mohammadi, Gelareh

A2 - Celiktutan, Oya

PB - Association for Computing Machinery (ACM)

CY - New York, NY

T2 - 27th International Conference on Multimodal Interaction, ICMI 2025

Y2 - 13 October 2025 through 17 October 2025

ER -

Li L. Multimodal Conversational Events Estimation in Complex Social Scenes. In Subramanian R, Nakano YI, Gedeon T, Kankanhalli M, Guha T, Shukla J, Mohammadi G, Celiktutan O, editors, ICMI 2025 - Proceedings of the 27th International Conference on Multimodal Interaction. New York, NY: Association for Computing Machinery (ACM). 2025. p. 716-720 doi: 10.1145/3716553.3750821

Multimodal Conversational Events Estimation in Complex Social Scenes

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this