Computer Says I Don’t Know: An Empirical Approach to Capture Moral Uncertainty in Artificial Intelligence

Andreia Martinho*, Maarten Kroesen, Caspar Chorus

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

2 Citations (Scopus)
49 Downloads (Pure)


As AI Systems become increasingly autonomous, they are expected to engage in decision-making processes that have moral implications. In this research we integrate theoretical and empirical lines of thought to address the matters of moral reasoning and moral uncertainty in AI Systems. We reconceptualize the metanormative framework for decision-making under moral uncertainty and we operationalize it through a latent class choice model. The core idea being that moral heterogeneity in society can be codified in terms of a small number of classes with distinct moral preferences and that this codification can be used to express moral uncertainty of an AI. Choice analysis allows for the identification of classes and their moral preferences based on observed choice data. Our reformulation of the metanormative framework is theory-rooted and practical in the sense that it avoids runtime issues in real time applications. To illustrate our approach we conceptualize a society in which AI Systems are in charge of making policy choices. While one of the systems uses a baseline morally certain model, the other uses a morally uncertain model. We highlight cases in which the AI Systems disagree about the policy to be chosen, thus illustrating the need to capture moral uncertainty in AI systems.

Original languageEnglish
Pages (from-to)215-237
Number of pages23
JournalMinds and Machines
Issue number2
Publication statusPublished - 2021


  • Artificial Intelligence
  • Discrete Choice Analysis
  • Latent Class Choice Model
  • Metanormative Theory
  • Morality
  • Uncertainty


Dive into the research topics of 'Computer Says I Don’t Know: An Empirical Approach to Capture Moral Uncertainty in Artificial Intelligence'. Together they form a unique fingerprint.

Cite this