Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing

Xingyu Zhao, Valentin Robu, David Flynn, Kizito Salako, Lorenzo Strigini

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

11 Citations (Scopus)

Abstract

There is an urgent societal need to assess whether autonomous vehicles (AVs) are safe enough. From published quantitative safety and reliability assessments of AVs, we know that, given the goal of predicting very low rates of accidents, road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing - knowledge which could bring substantial advantages if the AV design allows strong expectations of safety before road testing. We present the advantages of a new variant of Conservative Bayesian Inference (CBI), which uses prior knowledge while avoiding optimistic biases. We then study the trend of disengagements (take-overs by human drivers) by applying Software Reliability Growth Models (SRGMs) to data from Waymo's public road testing over 51 months, in view of the practice of software updates during this testing. Our approach is to not trust any specific SRGM, but to assess forecast accuracy and then improve forecasts. We show that, coupled with accuracy assessment and recalibration techniques, SRGMs could be a valuable test planning aid.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE 30th International Symposium on Software Reliability Engineering, ISSRE 2019
EditorsKatinka Wolter, Ina Schieferdecker, Barbara Gallina, Michel Cukier, Roberto Natella, Naghmeh Ivaki, Nuno Laranjeiro
PublisherIEEE
Pages13-23
Number of pages11
ISBN (Electronic)9781728149813
DOIs
Publication statusPublished - 2019
Externally publishedYes
Event30th IEEE International Symposium on Software Reliability Engineering, ISSRE 2019 - Berlin, Germany
Duration: 28 Oct 201931 Oct 2019

Publication series

NameProceedings - International Symposium on Software Reliability Engineering, ISSRE
Volume2019-October
ISSN (Print)1071-9458

Conference

Conference30th IEEE International Symposium on Software Reliability Engineering, ISSRE 2019
CountryGermany
CityBerlin
Period28/10/1931/10/19

Keywords

  • Autonomous vehicles
  • Conservative Bayesian inference
  • Reliability claims
  • Safety-critical systems
  • Software reliability growth model
  • Statistical testing
  • Ultra-high reliability

Fingerprint Dive into the research topics of 'Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing'. Together they form a unique fingerprint.

Cite this