Clarity is a worthwhile quality: On the role of task clarity in microtask crowdsourcing

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

57 Citations (Scopus)
6 Downloads (Pure)


Workers of microtask crowdsourcing marketplaces strive to find a balance between the need for monetary income and the need for high reputation. Such balance is often threatened by poorly formulated tasks, as workers attempt their execution despite a sub-optimal understanding of the work to be done. In this paper we highlight the role of clarity as a characterising property of tasks in crowdsourcing. We surveyed 100 workers of the CrowdFlower platform to verify the presence of issues with task clarity in crowdsourcing marketplaces, reveal how crowd workers deal with such issues, and motivate the need for mechanisms that can predict and measure task clarity. Next, we propose a novel model for task clarity based on the goal and role clarity constructs. We sampled 7.1K tasks from the Amazon mTurk marketplace, and acquired labels for task clarity from crowd workers. We show that task clarity is coherently perceived by crowd workers, and is affected by the type of the task. We then propose a set of features to capture task clarity, and use the acquired labels to train and validate a supervised machine learning model for task clarity prediction. Finally, we perform a long-term analysis of the evolution of task clarity on Amazon mTurk, and show that clarity is not a property suitable for temporal characterisation.

Original languageEnglish
Title of host publicationHT'17 Proceedings of the 28th ACM Conference on Hypertext and Social Media
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Number of pages10
ISBN (Electronic)978-1-4503-4708-2
Publication statusPublished - 2017
Event28th ACM Conference on Hypertext and Social Media, HT 2017 - Prague, Czech Republic
Duration: 4 Jul 20177 Jul 2017


Conference28th ACM Conference on Hypertext and Social Media, HT 2017
Country/TerritoryCzech Republic


  • Crowd Workers
  • Crowdsourcing
  • Goal Clarity
  • Microtasks
  • Performance
  • Prediction
  • Role Clarity
  • Task Clarity


Dive into the research topics of 'Clarity is a worthwhile quality: On the role of task clarity in microtask crowdsourcing'. Together they form a unique fingerprint.

Cite this