Welcome to MTurk Crowd

Work on Amazon Mechanical Turk, learn from the best, and have fun doing it. Join the crowd today!

Sign Up

MTurk bot article

Discussion in 'Forum Discussion' started by SJC, Sep 24, 2018.

  1. SJC

    SJC
    Expand Collapse
    Member

    Joined:
    Feb 17, 2016
    Messages:
    125
    Likes:
    168
  2. jan

    jan
    Expand Collapse
    Moderator

    • Moderator
    Joined:
    Jan 12, 2016
    Messages:
    9,166
    Likes:
    19,957
    https://blog.turkprime.com/after-th...h-data-collection-on-mturk-and-how-to-stop-it

    Our Investigation: What we did

    To learn about workers using server farms and how effective TurkPrime’s solution is at blocking them, we conducted a study which included a series of data quality checks. Our study examined response characteristics and data quality among workers known to operate through server farms (i.e., farmers) and workers known to operate without server farms (i.e., non-farmers). We refer to workers completing HITs from server farms as ‘farmers’ instead of ‘bots,’ because the term ‘farmer’ is agnostic as to whether these workers are humans, bots, or humans using some form of automation.
    and

    In addition to measures of data quality, we included several “bot-detection” measures in our study. Across these measures we found no evidence of bots—not one worker failed the re-captcha or honeypot. We did, however, find substantial evidence that farmers may be foreign workers operating outside the US. Farmers failed a basic English proficiency screener that 96% of non-farmers passed. In addition, while 95% of non-farmers referred to a pictured vegetable as an eggplant, 89% of farmers called the vegetable a ‘brinjal.’

    Consistent with the idea that farmers are foreign workers with low English proficiency, we found they do well on tasks that involve short instructions written in basic English. For example, farmers responded correctly to anchoring questions about the height of Mt. Everest and although they had a smaller effect size on the Mt. Everest anchoring task the manipulation was still effective for them. This evidence in combination with farmers’ apparent knowledge about the height of Mt. Everest—a mountain in Nepal, just to the north of India—strongly implies that these workers are humans based in India.
     
    jklmnop and KillMoles like this.
  3. sinon

    sinon
    Expand Collapse
    Watashi ga Kita!

    • Contributor
    Joined:
    Jan 12, 2016
    Messages:
    3,013
    Likes:
    7,106
    So basically all requesters need to do is put a location qual on their studies.
     
    Collapse Signature Expand Signature
    basketcasey likes this.