Disparate Impact Diminishes Consumer Trust Even for Advantaged Users

Tim Draws, Zoltán Szlávik, Benjamin Timmermans, Nava Tintarev, Kush R. Varshney, Michael Hind

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

Systems aiming to aid consumers in their decision-making (e.g., by implementing persuasive techniques) are more likely to be effective when consumers trust them. However, recent research has demonstrated that the machine learning algorithms that often underlie such technology can act unfairly towards specific groups (e.g., by making more favorable predictions for men than for women). An undesired disparate impact resulting from this kind of algorithmic unfairness could diminish consumer trust and thereby undermine the purpose of the system. We studied this effect by conducting a between-subjects user study investigating how (gender-related) disparate impact affected consumer trust in an app designed to improve consumers’ financial decision-making. Our results show that disparate impact decreased consumers’ trust in the system and made them less likely to use it. Moreover, we find that trust was affected to the same degree across consumer groups (i.e., advantaged and disadvantaged users) despite both of these consumer groups recognizing their respective levels of personal benefit. Our findings highlight the importance of fairness in consumer-oriented artificial intelligence systems.

Original languageEnglish
Title of host publicationPersuasive Technology - 16th International Conference, PERSUASIVE 2021, Proceedings
EditorsRaian Ali, Birgit Lugrin, Fred Charles
Place of PublicationCham
Pages135–149
Number of pages15
ISBN (Electronic)978-3-030-79459-0 978-3-030-79460-6
DOIs
Publication statusPublished - 2021
EventThe 16th International Conference on Persuasive Technologies - Bournemouth, United Kingdom
Duration: 12 Apr 202114 Apr 2021
https://persuasive2021.bournemouth.ac.uk/

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12684 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceThe 16th International Conference on Persuasive Technologies
CountryUnited Kingdom
CityBournemouth
Period12/04/2114/04/21
Internet address

Keywords

  • Algorithmic fairness
  • Consumer trust
  • Disparate impact

Fingerprint

Dive into the research topics of 'Disparate Impact Diminishes Consumer Trust Even for Advantaged Users'. Together they form a unique fingerprint.

Cite this