Abstract
There is an urgent societal need to assess whether autonomous vehicles (AVs) are safe enough. From published quantitative safety and reliability assessments of AVs, we know that, given the goal of predicting very low rates of accidents,
road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing – knowledge which could bring substantial advantages if the AV design allows strong expectations of safety before road testing. We present the advantages of a new variant of Conservative Bayesian Inference (CBI), which uses prior knowledge while avoiding optimistic biases. We then study the trend of disengagements (take-overs by human drivers) by applying Software Reliability Growth Models (SRGMs) to data from Waymo’s public road testing over 51 months, in view of the practice of software updates during this testing. Our approach is to not trust any specific SRGM, but to assess forecast accuracy
and then improve forecasts. We show that, coupled with accuracy assessment and recalibration techniques, SRGMs could be a valuable test planning aid.
road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing – knowledge which could bring substantial advantages if the AV design allows strong expectations of safety before road testing. We present the advantages of a new variant of Conservative Bayesian Inference (CBI), which uses prior knowledge while avoiding optimistic biases. We then study the trend of disengagements (take-overs by human drivers) by applying Software Reliability Growth Models (SRGMs) to data from Waymo’s public road testing over 51 months, in view of the practice of software updates during this testing. Our approach is to not trust any specific SRGM, but to assess forecast accuracy
and then improve forecasts. We show that, coupled with accuracy assessment and recalibration techniques, SRGMs could be a valuable test planning aid.
| Original language | English |
|---|---|
| Title of host publication | The 30th International Symposium on Software Reliability Engineering |
| Publication status | Accepted/In press - 20 Jul 2019 |
| Event | 30th International Symposium on Software Reliability Engineering 2019 - Berlin, Germany Duration: 28 Oct 2019 → 31 Oct 2019 |
Conference
| Conference | 30th International Symposium on Software Reliability Engineering 2019 |
|---|---|
| Abbreviated title | ISSRE 2019 |
| Country | Germany |
| City | Berlin |
| Period | 28/10/19 → 31/10/19 |
Keywords
- Autonomous vehicles
- software reliability
- Bayesian inference
- safety-critical systems
- reliability claims
- Statistical analysis
- Autonomous systems
- software reliability growth model
- ultra-high reliability