Just the right mood for HIT! Analyzing the role of worker moods in conversational microtask crowdsourcing

Sihang Qiu*, Ujwal Gadiraju, Alessandro Bozzon

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

6 Citations (Scopus)
18 Downloads (Pure)

Abstract

Conversational agents are playing an increasingly important role in providing users with natural communication environments, improving outcomes in a variety of domains in human-computer interaction. Crowdsourcing marketplaces are simultaneously flourishing, and it has never been easier to acquire large-scale human input from online workers. Recent works have revealed the potential of conversational interfaces in improving worker engagement and satisfaction. At the same time, worker moods have been shown to have significant effects on quality related outcomes. Little is known about the role of worker moods in shaping work in conversational microtask crowdsourcing. In this paper, we conducted a crowdsourcing study addressing 600 unique online workers, to investigate the role that worker moods play in conversational microtask crowdsourcing. We also explore whether suitable conversational styles of the agent can affect the performance of workers in different moods. Our results show that workers in a pleasant mood tend to produce significantly higher quality results (over 20%), exhibit greater engagement (an increase by around 19%) and report a lower cognitive load (by over 12%), and a suitable conversational style can have a significant impact on workers in different moods. Our findings advance the current understanding of conversational microtask crowdsourcing and have important implications on designing future conversational crowdsourcing systems.

Original languageEnglish
Title of host publicationWeb Engineering - 20th International Conference, ICWE 2020, Proceedings
EditorsMaria Bielikova, Tommi Mikkonen, Cesare Pautasso
Place of PublicationCham
PublisherSpringer
Pages381-396
Number of pages16
ISBN (Electronic)978-3-030-50578-3
ISBN (Print)978-3-030-50577-6
DOIs
Publication statusPublished - 2020
Event20th International Conference on Web Engineering, ICWE 2020 - Helsinki, Finland
Duration: 9 Jun 202012 Jun 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12128
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Web Engineering, ICWE 2020
Country/TerritoryFinland
CityHelsinki
Period9/06/2012/06/20

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • Conversational agent
  • Conversational style
  • Crowdsourcing
  • Moods
  • Worker moods
  • Worker performance

Fingerprint

Dive into the research topics of 'Just the right mood for HIT! Analyzing the role of worker moods in conversational microtask crowdsourcing'. Together they form a unique fingerprint.

Cite this