What do You Mean? Interpreting Image Classification with Crowdsourced Concept Extraction and Analysis

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

14 Citations (Scopus)
54 Downloads (Pure)

Abstract

Global interpretability is a vital requirement for image classification applications. Existing interpretability methods mainly explain a model behavior by identifying salient image patches, which require manual efforts from users to make sense of, and also do not typically support model validation with questions that investigate multiple visual concepts. In this paper, we introduce a scalable human-in-the-loop approach for global interpretability. Salient image areas identified by local interpretability methods are annotated with semantic concepts, which are then aggregated into a tabular representation of images to facilitate automatic statistical analysis of model behavior. We show that this approach answers interpretability needs for both model validation and exploration, and provides semantically more diverse, informative, and relevant explanations while still allowing for scalable and cost-efficient execution.

Original languageEnglish
Title of host publicationThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021
Pages1937-1948
Number of pages12
ISBN (Electronic)9781450383127
DOIs
Publication statusPublished - 2021
EventInternational World Wide Web Conference 2021 - Ljubljana, Slovakia
Duration: 19 Apr 202123 Apr 2021

Publication series

NameThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021

Conference

ConferenceInternational World Wide Web Conference 2021
Abbreviated titleWWW ’21
Country/TerritorySlovakia
CityLjubljana
Period19/04/2123/04/21

Keywords

  • Concept extraction
  • Human computation
  • Image classification
  • Machine-learning interpretability

Fingerprint

Dive into the research topics of 'What do You Mean? Interpreting Image Classification with Crowdsourced Concept Extraction and Analysis'. Together they form a unique fingerprint.

Cite this