Bayesian causal inference in automotive software engineering and online evaluation

Understand

Randomised field experiments, such as A/B testing, have long been the gold standard for evaluating software changes.

• In the automotive domain, running randomised field experiments is not always desired, possible, or even ethical.
• In the face of such limitations, we develop a framework BOAT (Bayesian causal modelling for ObvservAtional Testing), utilising observational studies in combination with Bayesian causal inference, in order to understand real-world impacts from complex automotive software updates and help software development organisations arrive at causal conclusions.
• In this study, we present three causal inference models in the Bayesian framework and their corresponding cases to address three commonly experienced challenges of software evaluation in the automotive domain.