SBM: Social Behavior Model for Human-Like Action Generation

Humans use verbal and nonverbal cues for effective communication, particularly during group interactions. Enabling intelligent systems — such as robots and virtual agents — to understand and generate such cues is crucial to facilitate natural and trustworthy human-robot interactions. We propose Social Behavior Model (SBM), a novel framework to generate socially appropriate actions in multiparty scenarios. Specifically, SBM takes into account the contextual information from surrounding individuals and the history of interaction data to generate socially coherent actions for an intelligent agent, including dialogue content and nonverbal cues like pose. To adapt pre-trained LLMs to the domain of social behavior, we fine-tune them using the Low-Rank Adaptation (LoRA) technique on a newly curated, labeled dataset containing multiparty social cues such as text and pose data. This method preserves the base model’s capabilities while enabling domain-specific adaptation with minimal computational cost. Given the lack of prior work on multiparty social behavior generation, we benchmark our model against state-of-the-art methods in dyadic pose generation. Our results demonstrate superior performance, establishing SBM as the first foundation model that integrates multiparty verbal and nonverbal social cues generation grounded in context understanding.