Third Party Tools to Enhance the Use of Mechanical Turk For Research



There are many third-party tools and extensions that help to extend the usability of Mechanical Turk for Research Purposes. Below are some of the tools we've come across.


Qualturk


Developed by René Kizilcec of Stanford's Lytics Lab, Qualturk (http://www.qualturk.com/, with more information and documentation at http://rene.kizilcec.com/software/) is a free tool to more tightly integrate the Qualtrics survey platform with Mechanical Turk.

Using QualTurk, researchers are able to simply implement effective screening measures, which prevent mturk workers not meeting the minimum qualifications for a study from participating. Qualifications include things like answering simple math problems (to ensure that workers are not answering randomly or being inattentive), taking too little time to finish the survey, and more.

The best part about Qualturk is that it is incredibly easy to set up. After registering for a free QualTurk account, you simply enter your Qualtrics survey link and the minimum time you think is reasonable for completing the survey into the site, copy HTML generated by the site into a Mechanical Turk HIT template, and launch the HIT! Filtering out participants who have been screened out once you've collected your results is also simple - QualTurk will automatically capture mturk worker IDs, and will show a "completion code" to participants who complete the survey and pass the screening. You can easily tell the people who have passed the screening by whether or not they have a completion code.

For more information and full instructions, sign up for a free account on qualturk.com.

Just Another Tool for Online Studies (JATOS)

Developed by Elisa Filevich and Kristian Lange, JATOS (http://www.jatos.org/) is an open source tool for setting up and running online studies implemented in javascript via mturk on a private server. It provides a GUI interface for implementing and collecting data from studies. Documentation is available on the site, and the developers have published a paper on PLOS One describing the tool: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0130834. 

From their paper:

 “Just Another Tool for Online Studies” (JATOS): an open source, cross-platform web application with a graphical user interface (GUI) that greatly simplifies setting up and communicating with a web server to host online studies that are written in JavaScript. JATOS is easy to install in all three major platforms (Microsoft Windows, Mac OS X, and Linux), and seamlessly pairs with a database for secure data storage. It can be installed on a server or locally, allowing researchers to try the application and feasibility of their studies within a browser environment, before engaging in setting up a server. All communication with the JATOS server takes place via a GUI (with no need to use a command line interface), making JATOS an especially accessible tool for researchers without a strong IT background. We describe JATOS’ main features and implementation and provide a detailed tutorial along with example studies to help interested researchers to set up their online studies. JATOS can be found under the Internet address: www.jatos.org.