This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics

T.A. Draws, N. Tintarev, Ujwal Gadiraju, Alessandro Bozzon, B. Timmermans

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

16 Citations (Scopus)
117 Downloads (Pure)

Abstract

In web search on debated topics, algorithmic and cognitive biases strongly influence how users consume and process information. Recent research has shown that this can lead to a search engine manipulation effect (SEME): when search result rankings are biased towards a particular viewpoint, users tend to adopt this favored viewpoint. To better understand the mechanisms underlying SEME, we present a pre-registered, 5 x 3 factorial user study investigating whether order effects (i.e., users adopting the viewpoint pertaining to higher-ranked documents) can cause SEME. For five different debated topics, we evaluated attitude change after exposing participants with mild pre-existing attitudes to search results that were overall viewpoint-balanced but reflected one of three levels of algorithmic ranking bias. We found that attitude change did not differ across levels of ranking bias and did not vary based on individual user differences. Our results thus suggest that order effects may not be an underlying mechanism of SEME. Exploratory analyses lend support to the presence of exposure effects (i.e., users adopting the majority viewpoint among the results they examine) as a contributing factor to users' attitude change. We discuss how our findings can inform the design of user bias mitigation strategies.

Original languageEnglish
Title of host publicationSIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
Place of PublicationNew York, NY, USA
PublisherAssociation for Computing Machinery (ACM)
Pages295–305
Number of pages11
ISBN (Electronic)9781450380379
DOIs
Publication statusPublished - 2021
Event44th International ACM SIGIR Conference on Research and Development in Information Retrieval - Online, Monteal, Canada
Duration: 11 Jul 202115 Jul 2021
https://sigir.org/sigir2021/

Publication series

NameSIGIR 2021 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

Conference44th International ACM SIGIR Conference on Research and Development in Information Retrieval
Abbreviated titleSIGIR 2021
Country/TerritoryCanada
CityMonteal
Period11/07/2115/07/21
Internet address

Keywords

  • ranking bias
  • user attitudes
  • user-centered evaluation
  • web search

Fingerprint

Dive into the research topics of 'This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics'. Together they form a unique fingerprint.

Cite this