SoK: Explainable Machine Learning for Computer Security Applications

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

1 Citation (Scopus)


Explainable Artificial Intelligence (XAI) aims to improve the transparency of machine learning (ML) pipelines. We systematize the increasingly growing (but fragmented) microcosm of studies that develop and utilize XAI methods for defensive and offensive cybersecurity tasks. We identify 3 cybersecurity stakeholders, i.e., model users, designers, and adversaries, who utilize XAI for 4 distinct objectives within an ML pipeline, namely 1) XAI-enabled user assistance, 2) XAI-enabled model verification, 3) explanation verification & robustness, and 4) offensive use of explanations. Our analysis of the literature indicates that many of the XAI applications are designed with little understanding of how they might be integrated into analyst workflows – user studies for explanation evaluation are conducted in only 14% of the cases. The security literature sometimes also fails to disentangle the role of the various stakeholders, e.g., by providing explanations to model users and designers while also exposing them to adversaries. Additionally, the role of model designers is particularly minimized in the security literature. To this end, we present an illustrative tutorial for model designers, demonstrating how XAI can help with model verification. We also discuss scenarios where interpretability by design may be a better alternative. The systematization and the tutorial enable us to challenge several assumptions, and present open problems that can help shape the future of XAI research within cybersecurity.
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P)
EditorsLisa O’Conner
Place of PublicationPiscataway
Number of pages20
ISBN (Electronic)978-1-6654-6512-0
ISBN (Print)978-1-6654-6513-7
Publication statusPublished - 2023
Event2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P) - Delft, Netherlands
Duration: 3 Jul 20237 Jul 2023
Conference number: 8th


Conference2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P)

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • XAI
  • Cybersecurity
  • Machine learning


Dive into the research topics of 'SoK: Explainable Machine Learning for Computer Security Applications'. Together they form a unique fingerprint.

Cite this