How robust is a robust policy? Comparing alternative robustness metrics for robust decision-making

Nowadays, decision-makers face deep uncertainties from a myriad of external factors such as climate change, population growth, new technologies, and economic developments. The challenge is to develop robust policies, which perform well across all possible resolutions of the uncertainties. One approach for achieving this is to design a policy to be adapted over time in response to how the future actually unfolds. A key determinant for the efficacy of such an adaptive policy is the specification of when and how to adapt it. This specification depends on how robustness is being operationalized. To date, there is little guidance for selecting an appropriate robustness metric. In this chapter we address this problem, using a case study of designing a policy for stimulating the transition of the European energy system towards more sustainable functioning using five different robustness metrics. We compare the policies as identified by each metric and discuss their relative merits. We highlight that the different robustness metrics emphasize different aspects of what makes a policy robust. More specifically, measures that separate dispersion and the mean, effectively doubling the number of objectives, provide very valuable information on the trade-offs between the mean performance of the policy and dispersion around this mean. We also discuss, based on our case, why analysts should use multiple robustness metrics.