A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable

Ruben S. Verhagen; Mark A. Neerincx; Myrthe L. Tielman

doi:10.1007/978-3-030-82017-6_8

A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable

Ruben S. Verhagen, Mark A. Neerincx, Myrthe L. Tielman

Interactive Intelligence

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

6 Citations (Scopus)

100 Downloads (Pure)

Abstract

Because of recent and rapid developments in Artificial Intelligence (AI), humans and AI-systems increasingly work together in human-agent teams. However, in order to effectively leverage the capabilities of both, AI-systems need to be understandable to their human teammates. The branch of eXplainable AI (XAI) aspires to make AI-systems more understandable to humans, potentially improving human-agent teamwork. Unfortunately, XAI literature suffers from a lack of agreement regarding the definitions of and relations between the four key XAI-concepts: transparency, interpretability, explainability, and understandability. Inspired by both XAI and social sciences literature, we present a two-dimensional framework that defines and relates these concepts in a concise and coherent way, yielding a classification of three types of AI-systems: incomprehensible, interpretable, and understandable. We also discuss how the established relationships can be used to guide future research into XAI, and how the framework could be used during the development of AI-systems as part of human-AI teams.

Original language	English
Title of host publication	Explainable and Transparent AI and Multi-Agent Systems
Subtitle of host publication	Third International Workshop, EXTRAAMAS 2021
Editors	Davide Calvaresi, Amro Najjar, Michael Winikoff, Kary Främling
Place of Publication	Cham
Publisher	Springer
Pages	119-138
Number of pages	20
ISBN (Electronic)	978-3-030-82017-6
ISBN (Print)	978-3-030-82016-9
DOIs	https://doi.org/10.1007/978-3-030-82017-6_8
Publication status	Published - 2021
Event	EXTRAAMAS 2021: Third International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems - Virtual at London, United Kingdom Duration: 3 May 2021 → 7 May 2021

Publication series

Name	Part of the Lecture Notes in Computer Science book series
Publisher	Springer
Volume	12688

Workshop

Workshop	EXTRAAMAS 2021
Country/Territory	United Kingdom
City	Virtual at London
Period	3/05/21 → 7/05/21

Bibliographical note

Accepted author manuscript

Keywords

Explainability
Explainable AI
Human-agent teaming
Interpretability
Transparency
Understandability

Access to Document

10.1007/978-3-030-82017-6_8

EXTRAAMAS2021_Verhagen_postprintAccepted author manuscript, 769 KB

Cite this

Verhagen, R. S., Neerincx, M. A., & Tielman, M. L. (2021). A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable. In D. Calvaresi, A. Najjar, M. Winikoff, & K. Främling (Eds.), Explainable and Transparent AI and Multi-Agent Systems: Third International Workshop, EXTRAAMAS 2021 (pp. 119-138). (Part of the Lecture Notes in Computer Science book series; Vol. 12688). Springer. https://doi.org/10.1007/978-3-030-82017-6_8

Verhagen, Ruben S. ; Neerincx, Mark A. ; Tielman, Myrthe L. / A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable. Explainable and Transparent AI and Multi-Agent Systems: Third International Workshop, EXTRAAMAS 2021. editor / Davide Calvaresi ; Amro Najjar ; Michael Winikoff ; Kary Främling. Cham : Springer, 2021. pp. 119-138 (Part of the Lecture Notes in Computer Science book series).

@inproceedings{fecd16c77ad54a919090a700a8915438,

title = "A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable",

abstract = "Because of recent and rapid developments in Artificial Intelligence (AI), humans and AI-systems increasingly work together in human-agent teams. However, in order to effectively leverage the capabilities of both, AI-systems need to be understandable to their human teammates. The branch of eXplainable AI (XAI) aspires to make AI-systems more understandable to humans, potentially improving human-agent teamwork. Unfortunately, XAI literature suffers from a lack of agreement regarding the definitions of and relations between the four key XAI-concepts: transparency, interpretability, explainability, and understandability. Inspired by both XAI and social sciences literature, we present a two-dimensional framework that defines and relates these concepts in a concise and coherent way, yielding a classification of three types of AI-systems: incomprehensible, interpretable, and understandable. We also discuss how the established relationships can be used to guide future research into XAI, and how the framework could be used during the development of AI-systems as part of human-AI teams.",

keywords = "Explainability, Explainable AI, Human-agent teaming, Interpretability, Transparency, Understandability",

author = "Verhagen, {Ruben S.} and Neerincx, {Mark A.} and Tielman, {Myrthe L.}",

note = "Accepted author manuscript; EXTRAAMAS 2021 : Third International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems ; Conference date: 03-05-2021 Through 07-05-2021",

year = "2021",

doi = "10.1007/978-3-030-82017-6_8",

language = "English",

isbn = "978-3-030-82016-9",

series = "Part of the Lecture Notes in Computer Science book series",

publisher = "Springer",

pages = "119--138",

editor = "Davide Calvaresi and Amro Najjar and Michael Winikoff and Kary Fr{\"a}mling",

booktitle = "Explainable and Transparent AI and Multi-Agent Systems",

address = "Germany",

}

Verhagen, RS , Neerincx, MA & Tielman, ML 2021, A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable. in D Calvaresi, A Najjar, M Winikoff & K Främling (eds), Explainable and Transparent AI and Multi-Agent Systems: Third International Workshop, EXTRAAMAS 2021. Part of the Lecture Notes in Computer Science book series, vol. 12688, Springer, Cham, pp. 119-138, EXTRAAMAS 2021, Virtual at London, United Kingdom, 3/05/21. https://doi.org/10.1007/978-3-030-82017-6_8

A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable. / Verhagen, Ruben S.; Neerincx, Mark A.; Tielman, Myrthe L.

Explainable and Transparent AI and Multi-Agent Systems: Third International Workshop, EXTRAAMAS 2021. ed. / Davide Calvaresi; Amro Najjar; Michael Winikoff; Kary Främling. Cham : Springer, 2021. p. 119-138 (Part of the Lecture Notes in Computer Science book series; Vol. 12688).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable

AU - Verhagen, Ruben S.

AU - Neerincx, Mark A.

AU - Tielman, Myrthe L.

N1 - Accepted author manuscript

PY - 2021

Y1 - 2021

N2 - Because of recent and rapid developments in Artificial Intelligence (AI), humans and AI-systems increasingly work together in human-agent teams. However, in order to effectively leverage the capabilities of both, AI-systems need to be understandable to their human teammates. The branch of eXplainable AI (XAI) aspires to make AI-systems more understandable to humans, potentially improving human-agent teamwork. Unfortunately, XAI literature suffers from a lack of agreement regarding the definitions of and relations between the four key XAI-concepts: transparency, interpretability, explainability, and understandability. Inspired by both XAI and social sciences literature, we present a two-dimensional framework that defines and relates these concepts in a concise and coherent way, yielding a classification of three types of AI-systems: incomprehensible, interpretable, and understandable. We also discuss how the established relationships can be used to guide future research into XAI, and how the framework could be used during the development of AI-systems as part of human-AI teams.

AB - Because of recent and rapid developments in Artificial Intelligence (AI), humans and AI-systems increasingly work together in human-agent teams. However, in order to effectively leverage the capabilities of both, AI-systems need to be understandable to their human teammates. The branch of eXplainable AI (XAI) aspires to make AI-systems more understandable to humans, potentially improving human-agent teamwork. Unfortunately, XAI literature suffers from a lack of agreement regarding the definitions of and relations between the four key XAI-concepts: transparency, interpretability, explainability, and understandability. Inspired by both XAI and social sciences literature, we present a two-dimensional framework that defines and relates these concepts in a concise and coherent way, yielding a classification of three types of AI-systems: incomprehensible, interpretable, and understandable. We also discuss how the established relationships can be used to guide future research into XAI, and how the framework could be used during the development of AI-systems as part of human-AI teams.

KW - Explainability

KW - Explainable AI

KW - Human-agent teaming

KW - Interpretability

KW - Transparency

KW - Understandability

UR - http://www.scopus.com/inward/record.url?scp=85113291407&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-82017-6_8

DO - 10.1007/978-3-030-82017-6_8

M3 - Conference contribution

SN - 978-3-030-82016-9

T3 - Part of the Lecture Notes in Computer Science book series

SP - 119

EP - 138

BT - Explainable and Transparent AI and Multi-Agent Systems

A2 - Calvaresi, Davide

A2 - Najjar, Amro

A2 - Winikoff, Michael

A2 - Främling, Kary

PB - Springer

CY - Cham

T2 - EXTRAAMAS 2021

Y2 - 3 May 2021 through 7 May 2021

ER -

Verhagen RS , Neerincx MA , Tielman ML. A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable. In Calvaresi D, Najjar A, Winikoff M, Främling K, editors, Explainable and Transparent AI and Multi-Agent Systems: Third International Workshop, EXTRAAMAS 2021. Cham: Springer. 2021. p. 119-138. (Part of the Lecture Notes in Computer Science book series). doi: 10.1007/978-3-030-82017-6_8

A Two-Dimensional Explanation Framework to Classify AI as Incomprehensible, Interpretable, or Understandable

Abstract

Publication series

Workshop

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this