Below you can find my slides:
The main points that I wanted to make:
- It is common to consider crowdsourcing as the "assembly line for knowledge work" and think of the workers as simple cogs in a big machine. It is almost a knee-jerk reaction to think negatively about the concept. However, it was the proper use of the assembly line (together with the proper automation) by Henry Ford that led to the first significant improvement in the level of living for the masses.
- Crowdsourcing suffers a lot due to significant worker turnover: Everyone who experimented with large tasks on MTurk knows that the participation distribution is very skewed. A few workers contribute the majority of the work, while a large number of workers contribute only minimally. Dealing with these hit-and-run workers is a pain, as we cannot apply any statistically meaningful mechanism for quality control.
- We ignore the fact that workers give back what they are given. Pay peanuts, get monkeys. Pay well, and get good workers. Needless to say, reputation and other quality signaling mechanisms are of fundamental importance for this task.
- Keeping the same workers around can give significant improvements in quality. Today on MTurk we have a tremendous turnover of workers, wasting significant effort and efficiencies. Whomever builds a strong base of a few good workers can pay the workers much better and, at the same time, generate a better product for lower cost than relying on an army of inexperienced, noisy workers.
Yes, at the end, crowdourcing is not about the crowd. It is about the individuals in the crowd. And we can now search for these valuable individuals very effectively. Crowdsourcing is crowdsearching.
