Although we tried for a while to use auto-granting mTurk qualifications to disallow workers from participating in an experiment multiple times, we found that the addition of a qualification discouraged workers from attempting to accept our HIT so drastically that batches which would have once finished in hours now took days to complete. As such, we've moved away from mTurk qualifications and towards using a combination of images and built-in features of the Qualtrics survey platform to discourage participating in experiments multiple times.
Most experimental surveys on mTurk look the same -- a bland consent form, or maybe just a link to an external survey. Nothing is distinctive, so workers may well not remember having previously participated in a survey. To combat this problem, we created a separate survey which was a single page with a picture on it, as well as text explaining that if participants recognized the picture, they had likely participated already. We used a whale, and the following text:
You may have already participated in this study, as our lab has run similar studies before. Some studies are completely independent, but others are related.
If you have already done this study, please return this HIT and do not do the experiment again – we cannot analyze data from someone who has already taken the experiment.
If you have seen a page like this before, and the image below, you have *definitely* participated in this study before.
We hope that the image is distinctive enough to be remembered by most workers, but of course, this method relies on the conscientiousness of workers. However, creating it as a separate survey has two advantages:
Although it is difficult to be certain whether workers have participated in an experiment multiple times, we believe that these measures have been effective in curbing multiple participation in our experiments.
Recruiting Research Participants via Mechanical Turk > How to prevent workers from participating in experiments multiple times >