Welcome to MTurk Crowd

Work on Amazon Mechanical Turk, learn from the best, and have fun doing it. Join the crowd today!

Sign Up

Are bots now an epidemic problem with HITS?

Discussion in 'MTurk Help & Resources' started by user9430961, Nov 9, 2018.

  1. user9430961

    user9430961
    Expand Collapse
    New Member

    Joined:
    Apr 30, 2018
    Messages:
    10
    Likes:
    1
    When I first started as a Turker back in April 2018, I had no problem getting and working HITS, yet over the past 3 or so months, I would guess about 25% of my HITS now require me to check off that I am a “human” or use CAPTCHA or have some form of puzzle type check to ensure that I am a human and not a bot completing a survey.

    Yet I was just wondering, is there now an epidemic problem with Amazon Turkers using a form of bot system to complete HITS?

    I have now done over a 1,000 HITS, yet with as many attention checks that I have found with a good percentage of my HITS, I can not image that someone has come up with a bot to do these various random type HITS, with the chance of the bot not being able to get pass the attention check(s) and the strong chance to the HIT being rejected.

    Yet I would guess that these bots seldom make mistakes and can out smart the attention check(s), so that the HITS that the bots complete do not make enough mistakes to get enough rejections to close/suspend an account with too many rejections.

    In my case, I take my HITS seriously, and strive to never get a rejection. “To me”, NOT getting a rejection is more important than whatever the HIT pays :) "To me", working a HIT is kinda like a game for me, where working the HIT and NOT getting a rejection is the fun that comes from working HITS and getting paid in the process for not getting a rejection. :)
     
  2. TSolo315

    TSolo315
    Expand Collapse
    SnapNCrackle

    • Moderator
    Joined:
    Jan 12, 2016
    Messages:
    3,218
    Likes:
    10,892
    A researcher noticed a pattern regarding junk responses he was getting: https://www.reddit.com/r/mturk/comments/95yy96/data_contamination/

    Made the assumption it was a bot issue and scared other researchers into adding bot prevention measures to their surveys,

    Turns out it was really foreign workers using VPNs just inputting nonsense: https://blog.turkprime.com/after-th...h-data-collection-on-mturk-and-how-to-stop-it

    So no, bots are not an epidemic. Making a bot to fill out surveys without looking like a bot would be very difficult and more trouble than it would be worth.
     
    Collapse Signature Expand Signature
  3. user9430961

    user9430961
    Expand Collapse
    New Member

    Joined:
    Apr 30, 2018
    Messages:
    10
    Likes:
    1
    TSolo315, thank you so very much for shedding light on this issue. OMG! :wideyed: Up until I read those articles (via the 2 links you provided), I had no idea that is what had happened.

    Thanks again. :happy:
     
  4. MICHAEL BUDAY

    MICHAEL BUDAY
    Expand Collapse
    New Member

    Joined:
    Dec 12, 2017
    Messages:
    12
    Likes:
    13
    I to started on MTurk December 2017, and have also noticed the amount of work way reduced since last May. I too have done over 2500 jobs and only have 19 rejections. I do everything I've read here to get more work, and to some degree am able to nonchalantly make $10.00 a day (just want to cover some bills), but I've heard about these "bots" and TSolo315 gives some thanks in making me feel a little better.
    I do know that there are a ton of us wanting to do jobs, and they always go way to quickly. But, what gets me is when a HIT goes up and is looking for 15 or more people, that those HITS vanquish quickly as well.
    I really hope there isn't anything shady going on, as it's nice working from home. And MTurk is a Godsend.
     
  5. user9430961

    user9430961
    Expand Collapse
    New Member

    Joined:
    Apr 30, 2018
    Messages:
    10
    Likes:
    1
    Hey there Michael. Congratulations on doing over 2,500 HITS! WOW!!!

    I know that there is still a TON of stuff I need to read about Amazon Turk, but I just don’t have the time . . . I really would just prefer to concentrate on making $5 to $7 a day, and I’m happy. Thus far, if I really apply myself, at least during the week (Monday - Friday), I usually can make that amount. Yet often I don’t bother Turking on the weekends, because “for me”, it is often hard to make $1 on a Saturday or Sunday (since there does not seem to be many HITS available (when I try to Turk), and I don't have the time to wait to stare at my computer for HITS to come in on a typical weekend . . . yet a few months ago, I was able to score a nice $5 HIT early one Saturday morning, I guess because there were not that many Turkers online at the time, so I was able to get the HIT.

    If I am not mistaken, I think I read in an article (before I started Turking), that there are various computer programs that Turkers can get (that can be customized) to automatically accept HITS when they are available . . . thus making it hard to manually get the ones that a person might want or making it harder to manually get a high paying one . . . thus with those programs, a HIT looking for multiple people are scooped-up in about 1 - 2 seconds. Yet I think with those computer programs, you can set it to automatically accept a HIT (for example) when it is more than $2 . . . thus those HITS disappear as soon as you see it.

    Yet I really don’t want a program gathering HITS for me, because I do not want to take a chance I miss or forgot one and they end up expiring . . . which may effect my overall performance record.

    Yet I do find it irksome that some HITS have tiny Attention Checks, that it seems like the requestor “REALLY” wants a Turker to fail, just so that there HIT is rejected. Out of all of the rejections I received (I only have 5 thus far, out of 1,175 HITS), it was always on a HIT that payed less than 40 cents. I often do not find such hard & stern types of trickery in HITS that pay over $1. As a rule, I always block a requester that rejects my work . . . to better avoid their possible tricks for future HITS. I try always be thorough with all of my work, but I hate rejections, and it seems like I have gotten less rejections by blocking the requestors that have issued me a rejection.

    Yet it really irks me, when they hid Attention Checks in the main information page of a HIT, where it might have a question at the end of the information page, but buried in the HIT information page, it will say something like “skip answering that question at the end of the page, and hit the arrow button to start the survey, to show us that you are paying attention”. I’m like, who has time to read an information page that will take 3 - 5 minutes to read, when the HIT only pays 25 cents, but will take 15 - 20 minutes to complete (AFTER) you read the main HIT information page. GRRRRR!!! :(

    Anyway Michael, that is my two cents worth. Yet maybe someone else will come along and shed more light on your post.

    Good luck to you. :happy:
     
  6. jan

    jan
    Expand Collapse
    Moderator

    • Moderator
    Joined:
    Jan 12, 2016
    Messages:
    9,478
    Likes:
    20,605
    In your post, you worry about not wanting hits to expire.

    Amazon used to collect that as a statistic and it was used, albeit rarely, by requesters. (Very, very rarely).
    It is no longer collected as a statistic. So, returned hits and expired hits are no problem :)
     
  7. jan

    jan
    Expand Collapse
    Moderator

    • Moderator
    Joined:
    Jan 12, 2016
    Messages:
    9,478
    Likes:
    20,605
    There are literally thousands of turkers on line at any moment - so while 15 hits disappearing all at once seems remarkable (as you watch your computer).

    It's not. The competition is fierce :)
     
  8. user9430961

    user9430961
    Expand Collapse
    New Member

    Joined:
    Apr 30, 2018
    Messages:
    10
    Likes:
    1
    Thank you Jan, I appreciate your help. I feel better now. :)

    Out of curiosity, when a HIT is returned, do they pop back up as an available HIT or does the requester have to resubmit it again, so that another Turker can work it?
     
  9. jan

    jan
    Expand Collapse
    Moderator

    • Moderator
    Joined:
    Jan 12, 2016
    Messages:
    9,478
    Likes:
    20,605
    it comes up as available. That's why we panda hits that are gone - so that we can catch them -if they are returned or expire for someone else :)
     
  10. user9430961

    user9430961
    Expand Collapse
    New Member

    Joined:
    Apr 30, 2018
    Messages:
    10
    Likes:
    1
    Thank you, Jan. I appreciate your help and great answer to my question. Thanks so much. :)
     
    jan likes this.