Toward a Learning Science for Complex Crowdsourcing Tasks / Shayan Doroudi, Ece Kamar and Emma Brunskill.
We explore how crowdworkers can be trained to tackle complex crowdsourcing tasks. We are particularly interested in training novice workers to perform well on solving tasks in situations where the space of strategies is large and workers need to discover and try different strategies to be successful...
Saved in:
| Online Access: |
Full Text (via ERIC) |
|---|---|
| Main Authors: | , , , |
| Format: | eBook |
| Language: | English |
| Published: |
[Place of publication not identified] :
Distributed by ERIC Clearinghouse,
2016.
|
| Subjects: |
MARC
| LEADER | 00000cam a22000002u 4500 | ||
|---|---|---|---|
| 001 | b8905739 | ||
| 003 | CoU | ||
| 005 | 20170209091047.0 | ||
| 006 | m o d f | ||
| 007 | cr ||||||||||| | ||
| 008 | 160507s2016 xx |||| ot ||| | eng d | ||
| 035 | |a (ERIC)ed566648 | ||
| 035 | |a (MvI) 2K000000546583 | ||
| 040 | |a ericd |b eng |c MvI |d MvI | ||
| 099 | |a ED566648 | ||
| 100 | 1 | |a Doroudi, Shayan. | |
| 245 | 1 | 0 | |a Toward a Learning Science for Complex Crowdsourcing Tasks / |c Shayan Doroudi, Ece Kamar and Emma Brunskill. |
| 264 | 1 | |a [Place of publication not identified] : |b Distributed by ERIC Clearinghouse, |c 2016. | |
| 300 | |a 1 online resource (12 pages) | ||
| 336 | |a text |b txt |2 rdacontent. | ||
| 337 | |a computer |b c |2 rdamedia. | ||
| 338 | |a online resource |b cr |2 rdacarrier. | ||
| 500 | |a Abstractor: As Provided. |5 ericd. | ||
| 516 | |a Text (Reports, Research) | ||
| 516 | |a Text (Speeches/Meeting Papers) | ||
| 520 | |a We explore how crowdworkers can be trained to tackle complex crowdsourcing tasks. We are particularly interested in training novice workers to perform well on solving tasks in situations where the space of strategies is large and workers need to discover and try different strategies to be successful. In a first experiment, we perform a comparison of five different training strategies. For complex web search challenges, we show that providing expert examples is an effective form of training, surpassing other forms of training in nearly all measures of interest. However, such training relies on access to domain expertise, which may be expensive or lacking. Therefore, in a second experiment we study the feasibility of training workers in the absence of domain expertise. We show that having workers validate the work of their peer workers can be even more effective than having them review expert examples if we only present solutions filtered by a threshold length. The results suggest that crowdsourced solutions of peer workers may be harnessed in an automated training pipeline. | ||
| 524 | |a Online Submission, Paper presented at the CHI Conference on Human Factors in Computing Systems (San Jose, CA, May 7-12, 2016). |2 ericd. | ||
| 650 | 0 | 7 | |a Training Methods. |2 ericd. |
| 650 | 0 | 7 | |a Novices. |2 ericd. |
| 650 | 0 | 7 | |a Outsourcing. |2 ericd. |
| 650 | 0 | 7 | |a Comparative Analysis. |2 ericd. |
| 650 | 0 | 7 | |a Online Searching. |2 ericd. |
| 650 | 0 | 7 | |a Expertise. |2 ericd. |
| 650 | 0 | 7 | |a Feasibility Studies. |2 ericd. |
| 650 | 0 | 7 | |a Accuracy. |2 ericd. |
| 650 | 0 | 7 | |a Statistical Analysis. |2 ericd. |
| 650 | 0 | 7 | |a Instructional Effectiveness. |2 ericd. |
| 650 | 0 | 7 | |a Nonparametric Statistics. |2 ericd. |
| 700 | 1 | |a Kamar, Ece, |e author. | |
| 700 | 1 | |a Brunskill, Emma, |e author. | |
| 700 | 1 | |a Horvitz, Eric, |e author. | |
| 856 | 4 | 0 | |u http://files.eric.ed.gov/fulltext/ED566648.pdf |z Full Text (via ERIC) |
| 907 | |a .b89057399 |b 07-06-22 |c 11-01-16 | ||
| 998 | |a web |b 11-01-16 |c f |d m |e - |f eng |g xx |h 0 |i 1 | ||
| 956 | |a ERIC | ||
| 999 | f | f | |i 5823c164-9645-5cc2-ac32-eb3e267dcf0c |s b8914968-47e3-5755-85f9-fef795779dff |
| 952 | f | f | |p Can circulate |a University of Colorado Boulder |b Online |c Online |d Online |e ED566648 |h Other scheme |i web |n 1 |