Improving Reactions to Rejection in Crowdsourcing through Self-Reflection

Tom Edixhoven, Sihang Qiu, Lucie Kuiper, Olivier Dikken, Gwennan Smitskamp, Ujwal Gadiraju

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

1 Citation (Scopus)

Abstract

In popular crowdsourcing marketplaces like Amazon Mechanical Turk, crowd workers complete tasks posted by requesters in return for monetary rewards. Task requesters are solely responsible for deciding whether to accept or reject submitted work. Rejecting work can directly affect the monetary reward of corresponding workers, and indirectly influence worker qualifications and their future work opportunities in the marketplace. Unexpected or unwarranted rejections therefore result in negative emotions and reactions among workers. Despite the high prevalence of rejections in crowdsourcing marketplaces, little research has explored ways to mitigate the negative emotional repercussions of rejections on crowd workers. Addressing this important research gap, we investigate whether introducing self-reflection at different stages after task execution can alleviate the emotional toll of rejection decisions on crowd workers. Our work is inspired by prior studies in psychology that have shown that self-reflection on negative personal experiences can positively affect one's emotion. To this end, we carried out an experimental study investigating the impact of explicit self-reflection on the emotions of rejected crowd workers. Results show that allowing workers to self-reflect on their delivered work, especially before receiving a rejection, has a significantly positive impact on their self-reported emotions in terms of valence and dominance. Our findings reveal that introducing a self-reflection stage before workers receive acceptance or rejection decisions on submitted work, can help in positively influencing the emotions of a worker. These findings have important design implications towards fostering a healthier requester-worker relationship and contributing towards the sustainability of the crowdsourcing marketplace.

Original languageEnglish
Title of host publicationWebSci 2021 - Proceedings of the 13th ACM Web Science Conference
PublisherAssociation for Computing Machinery (ACM)
Pages74-83
Number of pages10
ISBN (Electronic)9781450383301
DOIs
Publication statusPublished - 2021
Event13th ACM Web Science Conference, WebSci 2021 - Virtual, Online, United Kingdom
Duration: 21 Jun 202125 Jun 2021

Publication series

NameACM International Conference Proceeding Series

Conference

Conference13th ACM Web Science Conference, WebSci 2021
CountryUnited Kingdom
CityVirtual, Online
Period21/06/2125/06/21

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

  • crowdsourcing
  • emotion
  • microtask
  • mood
  • rejection
  • self-reflection

Fingerprint

Dive into the research topics of 'Improving Reactions to Rejection in Crowdsourcing through Self-Reflection'. Together they form a unique fingerprint.

Cite this