In this paper we present a new method for obtaining situation awareness via the automatic recognition of agent behaviours. In contrast to many other approaches, the presented method models different behaviour durations without using a fixed classification window, and does not require a distribution of behaviour durations. We introduce the Variable Window Layered Hidden Markov Model (VW-LHMM) as an extension of the LHMM to specifically address behaviours with irregular duration. We validate our approach by simulating three high-level behaviours within the harbour and coastline security domain. We compare performance against the LHMM and show that our approach provides a 10% improvement in classification accuracy, in addition to earlier classification. © 2010 The authors and IOS Press. All rights reserved.