Toward a Learning Science for Complex Crowdsourcing Tasks / Shayan Doroudi, Ece Kamar and Emma Brunskill.
We explore how crowdworkers can be trained to tackle complex crowdsourcing tasks. We are particularly interested in training novice workers to perform well on solving tasks in situations where the space of strategies is large and workers need to discover and try different strategies to be successful...
Saved in:
| Online Access: |
Full Text (via ERIC) |
|---|---|
| Main Authors: | , , , |
| Format: | eBook |
| Language: | English |
| Published: |
[Place of publication not identified] :
Distributed by ERIC Clearinghouse,
2016.
|
| Subjects: |
| Summary: | We explore how crowdworkers can be trained to tackle complex crowdsourcing tasks. We are particularly interested in training novice workers to perform well on solving tasks in situations where the space of strategies is large and workers need to discover and try different strategies to be successful. In a first experiment, we perform a comparison of five different training strategies. For complex web search challenges, we show that providing expert examples is an effective form of training, surpassing other forms of training in nearly all measures of interest. However, such training relies on access to domain expertise, which may be expensive or lacking. Therefore, in a second experiment we study the feasibility of training workers in the absence of domain expertise. We show that having workers validate the work of their peer workers can be even more effective than having them review expert examples if we only present solutions filtered by a threshold length. The results suggest that crowdsourced solutions of peer workers may be harnessed in an automated training pipeline. |
|---|---|
| Item Description: | Abstractor: As Provided. |
| Physical Description: | 1 online resource (12 pages) |
| Type of Computer File or Data Note: | Text (Reports, Research) Text (Speeches/Meeting Papers) |
| Preferred Citation of Described Materials Note: | Online Submission, Paper presented at the CHI Conference on Human Factors in Computing Systems (San Jose, CA, May 7-12, 2016). |