Just the right mood for HIT! Analyzing the role of worker moods in conversational microtask crowdsourcing

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

Conversational agents are playing an increasingly important role in providing users with natural communication environments, improving outcomes in a variety of domains in human-computer interaction. Crowdsourcing marketplaces are simultaneously flourishing, and it has never been easier to acquire large-scale human input from online workers. Recent works have revealed the potential of conversational interfaces in improving worker engagement and satisfaction. At the same time, worker moods have been shown to have significant effects on quality related outcomes. Little is known about the role of worker moods in shaping work in conversational microtask crowdsourcing. In this paper, we conducted a crowdsourcing study addressing 600 unique online workers, to investigate the role that worker moods play in conversational microtask crowdsourcing. We also explore whether suitable conversational styles of the agent can affect the performance of workers in different moods. Our results show that workers in a pleasant mood tend to produce significantly higher quality results (over 20%), exhibit greater engagement (an increase by around 19%) and report a lower cognitive load (by over 12%), and a suitable conversational style can have a significant impact on workers in different moods. Our findings advance the current understanding of conversational microtask crowdsourcing and have important implications on designing future conversational crowdsourcing systems.

Original languageEnglish
Title of host publicationWeb Engineering - 20th International Conference, ICWE 2020, Proceedings
EditorsMaria Bielikova, Tommi Mikkonen, Cesare Pautasso
Place of PublicationCham
PublisherSpringer
Pages381-396
Number of pages16
ISBN (Electronic)978-3-030-50578-3
ISBN (Print)978-3-030-50577-6
DOIs
Publication statusPublished - 2020
Event20th International Conference on Web Engineering, ICWE 2020 - Helsinki, Finland
Duration: 9 Jun 202012 Jun 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12128
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference20th International Conference on Web Engineering, ICWE 2020
CountryFinland
CityHelsinki
Period9/06/2012/06/20

Keywords

  • Conversational agent
  • Conversational style
  • Crowdsourcing
  • Moods
  • Worker moods
  • Worker performance

Fingerprint Dive into the research topics of 'Just the right mood for HIT! Analyzing the role of worker moods in conversational microtask crowdsourcing'. Together they form a unique fingerprint.

  • Cite this

    Qiu, S., Gadiraju, U., & Bozzon, A. (2020). Just the right mood for HIT! Analyzing the role of worker moods in conversational microtask crowdsourcing. In M. Bielikova, T. Mikkonen, & C. Pautasso (Eds.), Web Engineering - 20th International Conference, ICWE 2020, Proceedings (pp. 381-396). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12128 ). Springer. https://doi.org/10.1007/978-3-030-50578-3_26