SPATIAL: Practical AI Trustworthiness with Human Oversight

Abdul-Rasheed Ottun; Rasinthe Marasinghe; Toluwani Elemosho; Mohan Liyanage; Ashfaq Hussain Ahmed; Michell Boerger; Chamara Sandeepa; Thulitha Senevirathna; Aaron Yi Ding; null More Authors

doi:10.1109/ICDCS60910.2024.00138

SPATIAL: Practical AI Trustworthiness with Human Oversight

Abdul-Rasheed Ottun, Rasinthe Marasinghe, Toluwani Elemosho, Mohan Liyanage, Ashfaq Hussain Ahmed, Michell Boerger, Chamara Sandeepa, Thulitha Senevirathna, Aaron Yi Ding, More Authors

Information and Communication Technology

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

Abstract

We demonstrate SPATIAL, a proof-of-concept system that augments modern applications with capabilities to analyze trustworthy properties of AI models. The practical analysis of trustworthy properties is key to guaranteeing the safety of users and overall society when interacting with AI -driven applications. SPATIAL implements AI dashboards to introduce human-in-the-loop capabilities for the construction of AI models. SPATIAL allows different stakeholders to obtain quantifiable insights that characterize the decision making process of AI. This information can then be used by the stakeholders to comprehend possible issues that influence the performance of AI models, such that the issues can be resolved by human operators. Through rigorous benchmarks and experiments in a real-world industrial application, we demonstrate that SPATIAL can easily augment modern applications with metrics to gauge and monitor trustworthiness. However, this, in turn, increases the complexity of developing and maintaining the systems implementing AI. Our work paves the way towards augmenting modern applications with trustworthy AI mechanisms and human oversight approaches.

Original language	English
Title of host publication	2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)
Publisher	IEEE
Pages	1427-1430
Number of pages	4
ISBN (Electronic)	979-8-3503-8605-9
ISBN (Print)	979-8-3503-8606-6
DOIs	https://doi.org/10.1109/ICDCS60910.2024.00138
Publication status	Published - 2024
Event	44th IEEE International Conference on Distributed Computing Systems - Jersey City, United States Duration: 23 Jul 2024 → 26 Jul 2024 https://icdcs2024.icdcs.org/

Conference

Conference	44th IEEE International Conference on Distributed Computing Systems
Abbreviated title	ICDCS 2024
Country/Territory	United States
City	Jersey City
Period	23/07/24 → 26/07/24
Internet address	https://icdcs2024.icdcs.org/

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Artificial Intelligence
Fairness
Human Oversight
Industrial Use Cases
Practical Trustworthiness

Access to Document

10.1109/ICDCS60910.2024.00138

Cite this

@inproceedings{ed576556bcba42858f4644c864b8b09b,

title = "SPATIAL: Practical AI Trustworthiness with Human Oversight",

abstract = "We demonstrate SPATIAL, a proof-of-concept system that augments modern applications with capabilities to analyze trustworthy properties of AI models. The practical analysis of trustworthy properties is key to guaranteeing the safety of users and overall society when interacting with AI -driven applications. SPATIAL implements AI dashboards to introduce human-in-the-loop capabilities for the construction of AI models. SPATIAL allows different stakeholders to obtain quantifiable insights that characterize the decision making process of AI. This information can then be used by the stakeholders to comprehend possible issues that influence the performance of AI models, such that the issues can be resolved by human operators. Through rigorous benchmarks and experiments in a real-world industrial application, we demonstrate that SPATIAL can easily augment modern applications with metrics to gauge and monitor trustworthiness. However, this, in turn, increases the complexity of developing and maintaining the systems implementing AI. Our work paves the way towards augmenting modern applications with trustworthy AI mechanisms and human oversight approaches.",

keywords = "Artificial Intelligence, Fairness, Human Oversight, Industrial Use Cases, Practical Trustworthiness",

author = "Abdul-Rasheed Ottun and Rasinthe Marasinghe and Toluwani Elemosho and Mohan Liyanage and Ahmed, {Ashfaq Hussain} and Michell Boerger and Chamara Sandeepa and Thulitha Senevirathna and Ding, {Aaron Yi} and {More Authors}",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 44th IEEE International Conference on Distributed Computing Systems, ICDCS 2024 ; Conference date: 23-07-2024 Through 26-07-2024",

year = "2024",

doi = "10.1109/ICDCS60910.2024.00138",

language = "English",

isbn = "979-8-3503-8606-6",

pages = "1427--1430",

booktitle = "2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)",

publisher = "IEEE",

address = "United States",

url = "https://icdcs2024.icdcs.org/",

}

Ottun, A-R, Marasinghe, R, Elemosho, T, Liyanage, M, Ahmed, AH, Boerger, M, Sandeepa, C, Senevirathna, T, Ding, AY & More Authors 2024, SPATIAL: Practical AI Trustworthiness with Human Oversight. in 2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS). IEEE, pp. 1427-1430, 44th IEEE International Conference on Distributed Computing Systems, Jersey City, New Jersey, United States, 23/07/24. https://doi.org/10.1109/ICDCS60910.2024.00138

TY - GEN

T1 - SPATIAL

T2 - 44th IEEE International Conference on Distributed Computing Systems

AU - Ottun, Abdul-Rasheed

AU - Marasinghe, Rasinthe

AU - Elemosho, Toluwani

AU - Liyanage, Mohan

AU - Ahmed, Ashfaq Hussain

AU - Boerger, Michell

AU - Sandeepa, Chamara

AU - Senevirathna, Thulitha

AU - Ding, Aaron Yi

AU - More Authors, null

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2024

Y1 - 2024

N2 - We demonstrate SPATIAL, a proof-of-concept system that augments modern applications with capabilities to analyze trustworthy properties of AI models. The practical analysis of trustworthy properties is key to guaranteeing the safety of users and overall society when interacting with AI -driven applications. SPATIAL implements AI dashboards to introduce human-in-the-loop capabilities for the construction of AI models. SPATIAL allows different stakeholders to obtain quantifiable insights that characterize the decision making process of AI. This information can then be used by the stakeholders to comprehend possible issues that influence the performance of AI models, such that the issues can be resolved by human operators. Through rigorous benchmarks and experiments in a real-world industrial application, we demonstrate that SPATIAL can easily augment modern applications with metrics to gauge and monitor trustworthiness. However, this, in turn, increases the complexity of developing and maintaining the systems implementing AI. Our work paves the way towards augmenting modern applications with trustworthy AI mechanisms and human oversight approaches.

AB - We demonstrate SPATIAL, a proof-of-concept system that augments modern applications with capabilities to analyze trustworthy properties of AI models. The practical analysis of trustworthy properties is key to guaranteeing the safety of users and overall society when interacting with AI -driven applications. SPATIAL implements AI dashboards to introduce human-in-the-loop capabilities for the construction of AI models. SPATIAL allows different stakeholders to obtain quantifiable insights that characterize the decision making process of AI. This information can then be used by the stakeholders to comprehend possible issues that influence the performance of AI models, such that the issues can be resolved by human operators. Through rigorous benchmarks and experiments in a real-world industrial application, we demonstrate that SPATIAL can easily augment modern applications with metrics to gauge and monitor trustworthiness. However, this, in turn, increases the complexity of developing and maintaining the systems implementing AI. Our work paves the way towards augmenting modern applications with trustworthy AI mechanisms and human oversight approaches.

KW - Artificial Intelligence

KW - Fairness

KW - Human Oversight

KW - Industrial Use Cases

KW - Practical Trustworthiness

UR - http://www.scopus.com/inward/record.url?scp=85203159713&partnerID=8YFLogxK

U2 - 10.1109/ICDCS60910.2024.00138

DO - 10.1109/ICDCS60910.2024.00138

M3 - Conference contribution

AN - SCOPUS:85203159713

SN - 979-8-3503-8606-6

SP - 1427

EP - 1430

BT - 2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)

PB - IEEE

Y2 - 23 July 2024 through 26 July 2024

ER -

SPATIAL: Practical AI Trustworthiness with Human Oversight

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Embargoed Document

Fingerprint

Cite this