Recruiting Research Participants via Mechanical Turk



Video overview of the benefits of Mechanical Turk to run experiments

Scientific publications on Mechanical Turk

Paolacci, G., & Chandler, J. (2014). Inside the Turk Understanding Mechanical Turk as a Participant Pool. Current Directions in Psychological Science, 23(3), 184-188.

Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's Mechanical Turk A New Source of Inexpensive, Yet High-Quality, Data?Perspectives on Psychological Science6(1), 3-5.

Paolacci, G., Chandler, J., & Ipeirotis, P. (2010). Running Experiments on Amazon Mechanical Turk. Judgment and Decision Making5(5), 411-419.

Ahn, D., & Yo, N. (2011). Social science for penniesScience

Article in Behavior Research Methods

Online resources on Mechanical Turk

A go-to site aimed at the research community, that aggregates resources as well as having regular blogs.
    Sample blog posts: Emailing workers using Python, Screening participants using Qualtrics, The Fundamental AMT Articles.

PsiTurk - a platform that augments Mechanical Turk and makes it easier to conduct experiments, by preventing repeat participation.

How to set up Mechanical Turk in 5-10 minutes?


What to do if you need help right now?


A go-to site aimed at the research community, that aggregates resources as well as having regular blogs.
Sample blog posts: Emailing workers using Python, Screening participants using Qualtrics, The Fundamental AMT Articles.

Getting started using Mturk


These are some useful links for how Amazon Mechanical Turk can be used for asking people to participate in studies. 

Our old lab guidelines for running Mturk experiments


(1) Use a script in the HIT to pipe MturkIDs into Qualtrics survey URL
(2) Match participants' MturkIDs to a list kept in an embedded data field in Qualtrics survey flow


Creating your HIT

For most qualtrics HITs, you can use MTurk's built in "survey Link" HIT type. When creating an HIT, choose this option:


You can then simply paste in instructions and your survey link in the relevant places



Adding a Response Code

In the screenshot above, you'll notice that there is a text box for entering a "Survey Code". This code is given to workers after they complete the survey to verify that they completed it. Here's how to include a response code that is individualized for every turker.
This method involves entering the piped text code ${e://Field/ResponseID} in an end of survey message in qualtrics, then making your survey use that message upon completion. This will show users their qualtrics Response ID (a unique random identifier for each survey response generated by Qualtrics). 

First, create a new end-of-survey message in your library:


Then create the message. We usually use something simple:


And save it. Now, go into your survey options, and choose the end of survey message under the "survey termination" options:


And done! When turkers finish your survey they will see a code, which they then input into mturk to finish the HIT!



How to Post

The search bar in mturk, and what an HIT with an autogrant qualification might look like.

Using a pre-screening image survey


This idea is based on the notion that workers will remember an image better than a mturk requester or type of HIT. You'll set up a survey with an image that will redirect people to another survey. This is imperfect, but it has the added bonus of being able to redirect to any survey you would like. This method relies on appending an embedded data value to the survey URL -- for more information on this, see http://www.qualtrics.com/university/researchsuite/append-information-to-the-url

Creating the image survey


To do this, first create a new survey in Qualtrics. This survey only needs one block and does not need to have an actual question. The block should have an image (it doesn't matter what the image is, so long as it is distinctive and recognizable) and some text saying something along the lines of "If you recognize this image, please do not take this survey." 

Next, go into the survey flow.  The first thing you will want to do is add an embedded data block right at the very top with a single embedded data field. You can name the embedded data field anything you'd like, but it will be the variable that controls what survey people are sent to after clicking through your image survey (see screenshot below).

Next, for every value you want the embedded data field to have (with each value representing a separate survey you want to redirect to), create a conditional branch where the condition specifies what to do if the embedded data field is equal to that value (again, see screenshot). 

 

Under each branch, add an end of survey element, and hit the customize button. Select the "redirect to a URL" option, and paste the URL to the survey you want people to take into the box. See the screenshot below. Do this for every individual survey you want to use this particular image.


Using Bonuses

You can't pay someone directly, and so you can create a HIT with zero reward, and then bonus somebody who you want to compensate.

Intermediate level information


Amazon's site for requesters that gives an overview of Mturk
Amazon Mechanical Turk User Interface Guide for Requesters (step by step instructions & images on doing things)




Block Worker from taking same HIT multiple times (link)

Experimental Turk also has an aggregation of Academic Articles on Mturk

Useful tips from IT World article (including follow-up studies)

Specific Goals

How to Contact Mturk Workers, Do follow up experiment



The Developer Forum Search on this topic of how to contact Mturk Workers (and any others) was useful

Emailing workers using Python, where the MturkIDs can just be pasted in in a list

How to link Mturk with Qualtrics (Not recommended by Dave)

http://reactiontimes.wordpress.com/2012/02/08/connecting-mturk-to-qualtrics-to-scriptingrt/


Other resources

This website has a collection of programming tools built up around Mturk, and the owner is looking to open source it: Headlampresearch.com link



Other uses of Mturk
These links may give ideas for other ways that Mturk can be used: With a bit of creativity, it probably offers valuable solutions for subcontracting out other work that is involved in doing research, developing experiments, running a lab, finding resources, providing & assessing online education, & providing & assessing web-based resources for habit and health behavior change.

Using Amazon Mechanical Turk for Data Projects, journalism uses by ProPublica7