ContextBot: Improving Response Consistency in Crowd-Powered Conversational Systems for Affective Support Tasks

Yao Ma, Tahir Abbas, Ujwal Gadiraju

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

30 Downloads (Pure)

Abstract

Crowd-powered conversational systems (CPCS) solicit the wisdom of crowds to quickly respond to on-demand users' needs. The very factors that make this a viable solution - -such as the availability of diverse crowd workers on-demand - - also lead to great challenges. The ever-changing pool of online workers powering conversations with individual users makes it particularly difficult to generate contextually consistent responses from a single user's standpoint. To tackle this, prior work has employed conversational facts extracted by workers to maintain a global memory, albeit with limited success. Through a controlled experiment, we explored if a conversational agent, dubbed ContextBot, can provide workers with the required context on the fly for successful completion of affective support tasks in CPCS, and explore the impact of ContextBot on the response quality of workers and their interaction experience. To this end, we recruited workers (N=351) from the Prolific crowd-sourcing platform and carried out a 3×3 factorial between-subjects study. Experimental conditions varied based on (i) whether or not context was elicited and informed by motivational interviewing techniques (MI-adherent guidance, general guidance, and no guidance), and (ii) different conversational entry points for workers to produce responses (early, middle, and late). Our findings show that: (a) workers who entered the conversation earliest were more likely to produce highly consistent responses after interacting with ContextBot; (b) showed better user experience after they interacted with ContextBot with a long chat history to surf; (c) produced more professional responses as endorsed by psychologists; (d) and that interacting with ContextBot through task completion did not negatively impact workers' cognitive load. Our findings shed light on the implications of building intelligent interfaces for scaffolding strategies to preserve consistency in dialogue in CPCS.

Original languageEnglish
Title of host publicationHT 2023 - The 34th ACM Conference on Hypertext and Social Media
PublisherAssociation for Computing Machinery (ACM)
ISBN (Electronic)9798400702327
DOIs
Publication statusPublished - 2023
Event34th ACM Conference on Hypertext and Social Media, HT 2023 - Rome, Italy
Duration: 4 Sept 20238 Sept 2023

Publication series

NameHT 2023 - The 34th ACM Conference on Hypertext and Social Media

Conference

Conference34th ACM Conference on Hypertext and Social Media, HT 2023
Country/TerritoryItaly
CityRome
Period4/09/238/09/23

Keywords

  • Chatbots
  • Crowd-powered Conversational Systems
  • Dialogue Context
  • Motivational Interviewing
  • Real-time Crowdsourcing

Fingerprint

Dive into the research topics of 'ContextBot: Improving Response Consistency in Crowd-Powered Conversational Systems for Affective Support Tasks'. Together they form a unique fingerprint.

Cite this