HIT Management Software for Requesters

[clickhappier]

This is an extensive list of software/tools that MTurk requesters can use to help with their HIT management/programming, which I originally cataloged in July 2014. Most of these do still require a solid technical background to set up and use, though it's less work than programming your own API-accessing systems from scratch.

There are dozens of other MTurk-related projects on GitHub beyond these; these are the more broadly-applicable ones (vs specific to one experiment), and which appear to be more feature-complete and documented.


Retake Prevention
General HIT Management/Programming Tools - Misc Platforms/Languages
General HIT Management/Programming Tools - PHP
General HIT Management/Programming Tools - Python


Previous updates:
2014-07-21: Added Scala: AutoMan.
2014-07-22: Added PHP: Turk50.
2014-08-05: Added R: ExtraBonus_anon.R (Bulk Bonuses for Mechanical Turk Workers).
2014-11-02: Included the contents of my formerly-linked post listing online retake-prevention methods. Added Perl: Grant Mechanical Turk Bonuses Easier, and Python: mturk-utils. Added additional related links to Perl: Amazon Mechanical Turk SDK for Perl, and Python: mturk-python complete API mturkcore.py.
2015-01-25: Added a note about R: MTurkR's Wizard GUI.

 

[clickhappier]

Retake Prevention


My list of online retake-prevention methods/services:

There are several methods I've seen requesters use for checking for retakes, to make sure a worker ID hasn't taken a particular survey before during a previous posting of it, without using MTurk's evil blocking system.

• New qualification management option added Jul 23, 2014 that's more conducive to retake-prevention usage: "DoesNotExist". http://mechanicalturk.typepad.com/blog/2014/07/new-qualification-comparators-add-greater-flexibility-to-qualifications-.html So you could assign a custom qual 'Already Took Survey Yadayada' to everyone who's previously completed your HITs, and then set 'Already Took Survey Yadayada does not exist (has not been granted)' as a requirement for your HITs.
  
• Some requesters post a list of worker IDs who've taken previous postings of their survey and tell workers to search it for their ID before accepting - either in the preview page of the HIT, or in a document on Google Docs or elsewhere linked from the preview page.
  
• Using functionality within Qualtrics: http://thebehaviorallab.wordpress.com/2012/10/08/excluding-mturk-workers-from-surveys-in-qualtrics-and-elsewhere/ , http://experimentalturk.files.wordpress.com/2012/02/screening-amt-workers-on-qualtrics-5-2.pdf
  
• Detecting the Worker ID and using Javascript to compare it to a predefined list of previous takers: http://turktools.net/use/check.html
  
• Unique Turker service, created by a researcher at Cornell University: http://uniqueturker.myleott.com/ ; tips at http://www.tylerjohnburleigh.com/?p=321 , http://www.tylerjohnburleigh.com/?p=496
  
• Turkitron service, created by researchers at Georgia Institute of Technology and Victoria University of Wellington: http://turkitron.com/
  
• TurkCheck service, created by a researcher at Georgetown University: http://www.perceptionstudies.com/turkcheck/createlink.php ; tips at http://faculty.georgetown.edu/sjb247/tutorials/turkcheck/
  
• Plus one that would have to be hosted on your own server:
  
  
  Python: TurkGate - by Adam Darlow and Gideon Goldin, at Brown University - website - GitHub repository - support forum - experimentalturk blog post - tips from Gary Lupyan
  
  which can be augmented by:
  Python: TurkGateManager - by Pierce Edmiston, at University of Wisconsin - website - GitHub repository
  ​

 

[clickhappier]

General HIT Management/Programming Tools - Misc Platforms/Languages


Chrome Browser Extension: Mechanical Turk Requester On-Demand Iframes - by aaamp - Chrome Web Store page - reddit thread - MTG thread

Amazon Mechanical Turk Command Line Tools (CLT) (for Windows or Linux/Mac) - by Amazon AWS team - website - documentation - tips from Kathleen Tuite


C# .NET: Amazon Mechanical Turk SDK for .NET - by Amazon AWS team - CodePlex repository - AWS featured sample code page - forked as a NuGet package by Andy Sherwood: GitHub repository

Erlang: erlcrowd Crowd Computing API Library - by Gleb Peregud and Brian Buchanan - GitHub repository

Go: goamz - by Gustavo Niemeyer - website - Launchpad repository - support forum

Go: mtwerk client library - by Aditya Mukerjee, formerly at Cornell University NYC Tech - GitHub repository

Groovy: gwurk Grails plugin - by Joshua Introne, at Massachusetts Institute of Technology - GitHub repository

Java: Amazon Mechanical Turk SDK for Java - by Amazon AWS team - SourceForge repository - AWS featured sample code page - forked as a Maven package by Tommy Chheng: GitHub repository

Java: java_mturk_rest - by cclo7 - GitHub repository

Java: TurKit - by Greg Little et al, at Massachusetts Institute of Technology - website - Google Code repository - support forum

Java: TurkServer - by Andrew Mao et al, at Harvard University - GitHub repository - rewritten to run on the 'Meteor' web app server: GitHub repository - Meteor version website

JS: mmturkey external HIT submission library - by Long Ouyang, at Stanford University - GitHub repository

Node: amt Node API for Amazon Mechanical Turk - by Riccardo Volonterio - GitHub repository

Node: formious - by Christopher H. Brown, at University of Texas at Austin - GitHub repository

Node: mturk Mechanical Turk APi for Node - by Jeff Crouse - GitHub repository

Perl: Amazon Mechanical Turk SDK for Perl - by Amazon AWS team - SourceForge repository - GitHub repository archived from CPAN repository - AWS featured sample code page - fix for an error, and unofficial release incorporating it: CPAN repository

Perl: Grant Mechanical Turk Bonuses Easier - by Mat Morrison - blog post with code

R: ExtraBonus_anon.R (Bulk Bonuses for Mechanical Turk Workers) - by Vanessa Williamson, at Harvard University - website - described in her May 2014 paper

R: MTurkR - by Thomas J. Leeper, at Aarhus University, Denmark - website - GitHub repository (also CRAN mirror at GitHub) - CRAN repository - slightly forked by Solomon Messing: GitHub repository - tips from Solomon Messing - tips from Kyle Hamilton - tips from Thomas J. Leeper on using MTurkR to contact workers
* Note: now includes a Wizard GUI which can be used to contact workers through the MTurk messaging system, among other features; you still need to install the R programming language environment first to use this.

R: mturkr (appears to be a different implementation) - by Hadley Wickham, at Rice University - GitHub repository

Ruby: Clockwork Raven - by Ben Weissmann, Edwin Chen, and Dave Buchfuhrer, at Twitter Inc. - website - GitHub repository - minimal support forum - twitter blog post - mturk blog post

Ruby: RTurk Mechanical Turk Library - by Ryan Tate - GitHub repository

Ruby: Ruby Libraries for Amazon Web Services - by Amazon AWS team - AWS featured sample code page - official code posting was hosted on the defunct rubyforge.org, but in 2011 was archived by Mike Nelson of SkillSlate/TaskRabbit here: GitHub repository

Ruby: turducken - by Henk van der Veen - GitHub repository

Ruby: turk-rest-api aka Ask-Human - by Lee Butterman - website - GitHub repository

Ruby: Turkee - by Jim Jones - GitHub repository (example) - AWS featured sample code page

Ruby: turksurveyor - by Adam Kapelner, at University of Pennsylvania - GitHub repository

Scala: AutoMan - by Dan Barowy, Charlie Curtsinger, Emery Berger, and Andrew McGregor, at University of Massachusetts - website - GitHub repository - slideshare presentation - MTF thread

 

[clickhappier]

General HIT Management/Programming Tools - PHP


PHP: CodeIgniter Amazon Mturk Plugin - by Rohan - GitHub repository

PHP: Drupal Amazon Mechanical Turk Integration - by Daniel Zhou, formerly at University of Michigan - Drupal project - formerly used in openturk.org project

PHP: LegionTools - by Mitchell Gordon and Walter S. Lasecki, at University of Rochester - website - university page - GitHub repository

PHP: mTurk PHP API - by Rob Beckett, et al - SourceForge repository

PHP: mturk php library - by Sandip Limbachiya - SourceForge repository - AWS featured sample code page

PHP: mturk-php complete API - by Fredrick Brennan - GitHub repository

PHP: mturk-consent - by Jeffrey Bigham, at University of Rochester - GitHub repository

PHP: mturkCustomTestQual - by Grant Tepper - GitHub repository

PHP: php-mturk-api - by arner - GitHub repository

PHP: PHP-Turk library - by Nicolas Rosental - GitHub repository

PHP: Turk mass notify of workers - by Stan Zhuravlev - GitHub repository

PHP: Turk50 - by David J. Malan, at Harvard University - website - zip file - (Matthew Davidson at Cognilab has an apparently-unrelated work-in-progress, also in PHP and also called 'turk50', at this GitHub repository )

 

[clickhappier]

General HIT Management/Programming Tools - Python


Python: amtBonusScript - by Desmond Ong, at Stanford University - minimal website - GitHub repository

Python: boto AWS library for Python - by Mitch Garnaat, et al - website - GitHub repository - Google Project repository - AWS featured sample code page - tips from Pam Mueller and Jesse Chandler on using Boto to contact workers (pdf)

Python: CrowdForge - by Boris Smus - website - GitHub repository

Python: djurk - by Glen Jarvis - website - GitHub repository

Python: hitman - by Dmitry Kachaev - GitHub repository

Python: mechanical_turker assignment results comparison - by Shane Shifflett - GitHub repository

Python: mturk IPython Notebook - by Kelly O'Brien, at AWeber Communications - GitHub repository

Python: MTurk Results Parser - by Shlomo Shon Dalezman, at University of Michigan - GitHub repository

Python: mturk.py command-line tool for using Amazon Mechanical Turk - by Yotam Gingold - GitHub repository

Python: mturk-python complete API mturkcore.py - by Fredrick Brennan - GitHub repository - blog post

Python: mturk-utils - by Dave Kleinschmidt, at University of Rochester - Bitbucket repository

Python: mturkweb batch retriever - by Raynor Vliegendhart, at Delft University of Technology, Netherlands - GitHub repository

Python: psiTurk - by researchers at New York University Computation and Cognition Lab - website - GitHub repository

Python: Submiterator - by Dan Lassiter, at Stanford University - GitHub repository

Python: Textonic - by Steven Lehrburger et al, formerly at New York University - website - GitHub repository

Python: TurkAdmin bonus/messaging tool for Google App Engine - by John Joseph Horton, at New York University - GitHub repository

Python: turkflow - by Fil Krynicki - GitHub repository - PyPI repository

Python: TurkLime LimeSurvey tool for Google App Engine - by John Joseph Horton, at New York University - GitHub repository

Python: turktools - by Michael Yoshitaka Erlewine, at Massachusetts Institute of Technology - website - GitHub repository

Python: utility (Mechanical Turk web framework) - by Michael Toomim, formerly at University of Washington - GitHub repository