Using work domain analysis to evaluate the impact of technological change on the performance of complex socio-technical systems

Complex socio-technical systems are notoriously difficult to evaluate.
The relationships between measurable processes and high-level descriptions of
performance are neither clear nor well understood. Consequently, it is extremely
difficult to relate changes made to physical components to overall system
performance. Using the relationships captured in work domain analysis,
this paper introduces a new approach for evaluating the impact of system
changes. An event and actor-independent model is used to describe the given
domain at a number of levels of abstraction. The relationship between these
descriptions forms the basis of this approach. A case study of a recently
upgraded UK land military headquarters is used to demonstrate the approach.
As the presented case study will show, a clear link can be made between
system changes that result in an increase in overall system performance, as well
as changes that result in a degradation. It is contended that the described
approach adds important structure to the way that evaluations of complex
socio-technical systems are considered, creating a traceable link between
changes made at a component level to high-order descriptions of domain
purpose.