Mixture of Switching Linear Dynamics to Discover Behavior Patterns in Object Tracks

Research output: Contribution to journalArticleScientificpeer-review

9 Citations (Scopus)


We present a novel non-parametric Bayesian model to jointly discover the dynamics of low-level actions and high-level behaviors of tracked objects. In our approach, actions capture both linear, low-level object dynamics, and an additional spatial distribution on where the dynamic occurs. Furthermore, behavior classes capture high-level temporal motion dependencies in Markov chains of actions, thus each learned behavior is a switching linear dynamical system. The number of actions and behaviors is discovered from the data itself using Dirichlet Processes. We are especially interested in cases where tracks can exhibit large kinematic and spatial variations, e.g. person tracks in open environments, as found in the visual surveillance and intelligent vehicle domains. The model handles real-valued features directly, so no information is lost by quantizing measurements into 'visual words', and variations in standing, walking and running can be discovered without discrete thresholds. We describe inference using Markov Chain Monte Carlo sampling and validate our approach on several artificial and real-world pedestrian track datasets from the surveillance and intelligent vehicle domain. We show that our model can distinguish between relevant behavior patterns that an existing state-of-the-art hierarchical model for clustering and simpler model variants cannot. The software and the artificial and surveillance datasets are made publicly available for benchmarking purposes.

Original languageEnglish
Article number7121003
Pages (from-to)322-334
Number of pages13
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number2
Publication statusPublished - 2016
Externally publishedYes


  • Hierarchical non-parametric graphical model
  • Human behavior analysis
  • Switching Linear Dynamical Systems


Dive into the research topics of 'Mixture of Switching Linear Dynamics to Discover Behavior Patterns in Object Tracks'. Together they form a unique fingerprint.

Cite this