Metrics for Evaluating Explainable Recommender Systems

Joris Hulstijn*, Igor Tchappi, Amro Najjar, Reyhan Aydoğan

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

13 Downloads (Pure)


Recommender systems aim to support their users by reducing information overload so that they can make better decisions. Recommender systems must be transparent, so users can form mental models about the system’s goals, internal state, and capabilities, that are in line with their actual design. Explanations and transparent behaviour of the system should inspire trust and, ultimately, lead to more persuasive recommendations. Here, explanations convey reasons why a recommendation is given or how the system forms its recommendations. This paper focuses on the question how such claims about effectiveness of explanations can be evaluated. Accordingly, we investigate various models that are used to assess the effects of explanations and recommendations. We discuss objective and subjective measurement and argue that both are needed. We define a set of metrics for measuring the effectiveness of explanations and recommendations. The feasibility of using these metrics is discussed in the context of a specific explainable recommender system in the food and health domain.

Original languageEnglish
Title of host publicationExplainable and Transparent AI and Multi-Agent Systems - 5th International Workshop, EXTRAAMAS 2023, Revised Selected Papers
EditorsDavide Calvaresi, Amro Najjar, Andrea Omicini, Rachele Carli, Giovanni Ciatto, Reyhan Aydogan, Yazan Mualla, Kary Främling
Number of pages19
ISBN (Print)9783031408779
Publication statusPublished - 2023
EventProceedings of the 5th International Workshop on EXTRAAMAS 2023 - London, United Kingdom
Duration: 29 May 202329 May 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14127 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceProceedings of the 5th International Workshop on EXTRAAMAS 2023
Country/TerritoryUnited Kingdom

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Evaluation
  • Explainable AI
  • Metrics
  • Recommender systems


Dive into the research topics of 'Metrics for Evaluating Explainable Recommender Systems'. Together they form a unique fingerprint.

Cite this