Explainable Cross-Topic Stance Detection for Search Results

Tim Draws, Karthikeyan Natesan Ramamurthy, Ioana Baldini, Amit Dhurandhar, Inkit Padhi, Benjamin Timmermans, Nava Tintarev

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

7 Citations (Scopus)
118 Downloads (Pure)

Abstract

One way to help users navigate debated topics online is to apply stance detection in web search. Automatically identifying whether search results are against, neutral, or in favor could facilitate diversification efforts and support interventions that aim to mitigate cognitive biases. To be truly useful in this context, however, stance detection models not only need to make accurate (cross-topic) predictions but also be sufficiently explainable to users when applied to search results - an issue that is currently unclear. This paper presents a study into the feasibility of using current stance detection approaches to assist users in their web search on debated topics. We train and evaluate 10 stance detection models using a stance-annotated data set of 1204 search results. In a preregistered user study (N = 291), we then investigate the quality of stance detection explanations created using different explainability methods and explanation visualization techniques. The models we implement predict stances of search results across topics with satisfying quality (i.e., similar to the state-of-the-art for other data types). However, our results reveal stark differences in explanation quality (i.e., as measured by users' ability to simulate model predictions and their attitudes towards the explanations) between different models and explainability methods. A qualitative analysis of textual user feedback further reveals potential application areas, user concerns, and improvement suggestions for such explanations. Our findings have important implications for the development of user-centered solutions surrounding web search on debated topics.

Original languageEnglish
Title of host publicationCHIIR 2023 - Proceedings of the 2023 Conference on Human Information Interaction and Retrieval
Place of PublicationNew York, NY, USA
PublisherAssociation for Computing Machinery (ACM)
Pages221-235
Number of pages15
ISBN (Electronic)979-8-4007-0035-4
DOIs
Publication statusPublished - 2023
Event8th ACM SIGIR Conference on Human Information Interaction and Retrieval, CHIIR 2023 - Austin, United States
Duration: 19 Mar 202323 Mar 2023

Publication series

NameCHIIR 2023 - Proceedings of the 2023 Conference on Human Information Interaction and Retrieval

Conference

Conference8th ACM SIGIR Conference on Human Information Interaction and Retrieval, CHIIR 2023
Country/TerritoryUnited States
CityAustin
Period19/03/2323/03/23

Keywords

  • bias
  • explainability
  • stance detection
  • viewpoint
  • web search

Fingerprint

Dive into the research topics of 'Explainable Cross-Topic Stance Detection for Search Results'. Together they form a unique fingerprint.

Cite this