My previous post about soliciting reviews using Mechanical Turk generated some reactions by the companies that were listed in the post. It seems that there is a perceived stigma associated with populating a website using paid reviews and companies feel obliged to either clarify that they are soliciting unbiased reviews, even though that was clear from the description of the task posted on Mechanical Turk.
This got me thinking: Why do we consider paid reviews to be inferior compared to their unpaid counterparts?
Yes, most of today's websites that are hosting reviews for anything under the sun would love to have armies of users contributing voluntarily their reviews. There is however a thing called "network effects" that require the website to have some content to attract users, that will in turn write additional reviews. But do you get the reviews to start with? Paying for them seems to be a legitimate approach, in my opinion.
Before the advent of the web, most of the "reviewers" (aka critics) were paid by their publishers to evaluate objectively the products in their area of interest. Nobody complained for the fact that the critics were being paid to write the reviews.
The complaints were coming from biases of the reviewer, or (even worse) when the reviewer was paid to write a non-objective review, effectively deceiving the readers.
So, it is the same game with Mechanical Turk. Nothing wrong with soliciting reviews in exchange for payment. However, it should be clear from the requester that the reviews have to be unbiased and should reflect the experience of the reviewer.
The outrage for Belkin was not due to the solicitation of reviews on Mechanical Turk. It was due to the solicitation of biased, positive reviews for a product that seemed to be a flop, which was equivalent to deceiving Belkin's customers.
