Wednesday, March 19, 2008

Mechanical Turk: The Demographics

Update: The results in this blog post are now obsolete. Please read the results of the new survey.

One of the common misbeliefs about Mechanical Turk is that it is a virtual sweatshop, essentially taking advantage of poor people in third world countries that are doing tedious tasks for pennies. Therefore, many people are afraid of outsourcing research tasks on Mechanical Turk, being afraid that the results will be either of very poor quality, or they will not be representative of the actual U.S. population.

Those that read the previous, qualitative survey about Mechanical Turk would have realized that the profile of the typical Turker is not of a person that completes tasks for a living in a developing country. Instead, Mechanical Turk tends to be often a replacement for TV, or simple something to spend some free time and get some spare cash in reward.

The next survey that I conducted focused more on the demographics of the Turkers. Are they uneducated, unemployed people with no income? Well, as you will see below the Turkers are a pretty representative sample of the online population, perhaps with a slight bias towards females and towards young participants. (See a detailed comparison on how the demographics of Mechanical Turk users compare to general demographics of Internet users)
Let's see the main results!

First, I would start with the country breakdown.

United States 76.25%
India 8.03%
United Kingdom 3.34%
Canada 2.34%

The clear result is that most of the participants are coming from the US and not from a third world country, despite the common misconception. This is due to the fact that in order to get paid, someone has to have a US bank account, or be willing to be paid using Amazon gift certificates.

Then, the gender breakdown:
As you can see, there are slightly more females that males. I do not have a definite reason yet, but I get a feeling that females are less inclined to "waste time" and find that if they can exploit their spare time to get a little bit of income, then they would do it.

Next, the age distribution:

Not surprisingly, many young people participate on Mechanical Turk, mainly as a way to get some extra cash and to be able to drive their car, get some items from Amazon and so on. (The Mechanical Turk payment can be either deposited in a US bank, or be given as an Amazon gift certificate.)

And what about education?

Turkers are a pretty representative sample. Most of them have a college education, and some of them even have PhDs! In fact, the distribution seems pretty similar to the distribution for the overall US population.

Similarly, the income distribution also follows closely the income distribution in the US:

Finally, why people participate in Mechanical Turk? From the qualitative survey, you could see that most of the participants mention money, one way or another. However, very few participate only for the money. (See also the detailed responses.) Here is the breakdown of the responders when they had to choose (not exclusively) between the choices "for money," "for fun," and "for killing time":

I hope that the results above shed some light. I have to thank my student Beibei for preparing the running the survey for me. The next steps now are to present the results of the qualitative survey in a coded/tabulated manner, and to give more details about the different tasks that we had run on Mechanical Turk and the lessons that we learned.

If you have any more questions that you would like to see answered, let me know!

See also