Multi-modal human aggression detection

J. F.P. Kooij, M. C. Liem, J. D. Krijnders, T.C. Andringa, D. M. Gavrila

Research output: Contribution to journalArticleScientificpeer-review

31 Citations (Scopus)


This paper presents a smart surveillance system named CASSANDRA, aimed at detecting instances of aggressive human behavior in public environments. A distinguishing aspect of CASSANDRA is the exploitation of complementary audio and video cues to disambiguate scene activity in real-life environments. From the video side, the system uses overlapping cameras to track persons in 3D and to extract features regarding the limb motion relative to the torso. From the audio side, it classifies instances of speech, screaming, singing, and kicking-object. The audio and video cues are fused with contextual cues (interaction, auxiliary objects); a Dynamic Bayesian Network (DBN) produces an estimate of the ambient aggression level. Our prototype system is validated on a realistic set of scenarios performed by professional actors at an actual train station to ensure a realistic audio and video noise setting.

Original languageEnglish
Pages (from-to)106-120
Number of pages15
JournalComputer Vision and Image Understanding
Publication statusPublished - 2016
Externally publishedYes


  • Aggression detection
  • Automated video surveillance
  • Dynamic Bayesian Network
  • Multi-modal sensor fusion


Dive into the research topics of 'Multi-modal human aggression detection'. Together they form a unique fingerprint.

Cite this