Testing the reliability of accident analysis methods: a comparison of AcciMap, STAMP-CAST and AcciNet

Adam Hulme, Neville A. Stanton, Guy H. Walker, Patrick Waterson, Paul M. Salmon

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
15 Downloads (Pure)

Abstract

Accident analysis methods are used to model the multifactorial cause of adverse incidents. Methods such as AcciMap, STAMP-CAST and recently AcciNet, are systemic approaches that support the identification of safety interventions across sociotechnical system levels. Despite their growing popularity, little is known about how reliable systems-based methods are when used to describe, model and classify contributory factors and relationships. Here, we conducted an intra-rater and inter-rater reliability assessment of AcciMap, STAMP-CAST and AcciNet using the Signal Detection Theory (SDT) paradigm. A total of 180 hours’ worth of analyses across 360 comparisons were performed by 30 expert analysts. Findings revealed that all three methods produced a weak to moderate positive correlation coefficient, however the inter-rater reliability of STAMP-CAST was significantly higher compared to AcciMap and AcciNet. No statistically significant or practically meaningful differences were found between methods in the overall intra-rater reliability analyses. Implications and future research directions are discussed.
Original languageEnglish
Pages (from-to)1-21
Number of pages21
JournalErgonomics
Early online date31 Jul 2023
DOIs
Publication statusE-pub ahead of print - 31 Jul 2023

Keywords

  • AcciMap
  • Accident analysis
  • STAMP-CAST
  • reliability
  • systems

ASJC Scopus subject areas

  • Human Factors and Ergonomics
  • Physical Therapy, Sports Therapy and Rehabilitation

Cite this