A general method for human activity recognition in video

Neil Robertson, Ian D. Reid

Research output: Contribution to journalArticlepeer-review

136 Citations (Scopus)


In this paper we develop a system for human behaviour recognition in video sequences. Human behaviour is modelled as a stochastic sequence of actions. Actions are described by a feature vector comprising both trajectory information (position and velocity), and a set of local motion descriptors. Action recognition is achieved via probabilistic search of image feature databases representing previously seen actions. Hidden Markov Models (HMM) which encode scene rules are used to smooth sequences of actions. High-level behaviour recognition is achieved by computing the likelihood that a set of predefined HMMs explains the current action sequence. Thus, human actions and behaviour are represented using a hierarchy of abstraction: from person-centred actions, to actions with spatio-temporal context, to action sequences and, finally, general behaviours. While the upper levels all use Bayesian networks and belief propagation, the lowest level uses non-parametric sampling from a previously learned database of actions. The combined method represents a general framework for human behaviour modelling. We demonstrate results from broadcast tennis sequences and surveillance footage for automated video annotation.

Original languageEnglish
Pages (from-to)232–248
JournalComputer Vision and Image Understanding
Issue number2-3
Publication statusPublished - Nov 2006


  • Visual surveillance
  • Human activity recognition
  • Video annotation


Dive into the research topics of 'A general method for human activity recognition in video'. Together they form a unique fingerprint.

Cite this