ACMuse: An Actionable Multi-Stage Safety Evaluation Framework for Autonomous Vehicles

Safety evaluation methods for autonomous vehicles (AVs) have been recently studied. Existing methods such as sample-based methods employ criticality metrics to evaluate AV safety. They are typically restricted to single-dimension evaluations and fail to offer substantial evidence to address the critical question of how safe is safe enough. In response, we introduce an actionable multi-stage safety evaluation framework (ACMuse) designed to evaluate AV safety. This framework systematically measures safety performance through five sequential stages: safety distance assessment, collision risk check, collision avoidance capabilities assessment, collision severity assessment, and accident liability determination. Each stage is featured with specific quantitative indexes to enable an actionable framework across different test scenarios. We demonstrated the effectiveness of ACMuse by applying it to an actual Highway Pilot (HWP) system using real-world experiments. The results indicate that ACMuse offers deep and insightful safety analysis of the HWP, underscoring its ability to identify safety issues and its contributions to AV release.