Welcome to MTurk Crowd

Work on Amazon Mechanical Turk, learn from the best, and have fun doing it. Join the crowd today!

Sign Up
  1. MTurk's Worker website has changed! More info here: Worker is Coming: Discussion Mega Thread
    Dismiss Notice

Rep Present Feedback request for educational challenge entry verification

Discussion in 'Requesters' started by Dave Honess, Nov 14, 2017.

  1. Dave Honess

    Dave Honess
    Expand Collapse
    New Member

    • Requester
    Joined:
    Nov 14, 2017
    Messages:
    4
    Likes:
    11
    Hi everyone!

    I'm the programme manager for Astro Pi at the Raspberry Pi Foundation in Cambridge UK. Astro Pi is an annual science and coding competition where student-written code is run on the International Space Station to perform scientific experiments. It began as one of the educational projects for British ESA astronaut Tim Peake in 2015, open to the UK only, and was extended to all ESA member states for the flight of French ESA astronaut Thomas Pesquet last year. This year astronauts Paolo Nespoli and Alexander Gerst are the ambassadors.

    I'm using MTurk to verify that the entries for one of our challenges, Mission Zero, have obeyed the rules.

    I recently became a requester for the first time and uploaded 30 of these entries, one entry per HIT with 3 assignments each, as a test.

    Without any further explanation I'd like to see how people find the instructions I've provided, is everything clear? When the challenge is over on the 26th, I'll be uploading probably around 1000 more of these.

    s3.amazonaws.com/mturk_bulk/hits/291180703/rzWxRlzdUgVpYyZSm_eH0Q.html

    Many thanks!

    Dave
     
    #1 Dave Honess, Nov 14, 2017
    Last edited: Nov 14, 2017
    <Gucci>, jan, ducky and 1 other person like this.
  2. turker

    turker
    Expand Collapse
    How many pork chop of pork chop are in this image?

    • Contributor
    Joined:
    Jan 23, 2016
    Messages:
    11,789
    Likes:
    23,747
    greeting was a sideways mushroom followed by the words "good luck". so if I was doing those hits, I don't know if I should put yes or no for greeting. also I couldn't see the example of an error. these look easily doable if those 2 things are more clear.
     
    Collapse Signature Expand Signature
    <Gucci> and Dave Honess like this.
  3. kryss

    kryss
    Expand Collapse
    carburante al fuoco

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    19,230
    Likes:
    55,237
    Instructions seem clear to me.

    Thanks for reaching out!
     
    Collapse Signature Expand Signature
    <Gucci> and Dave Honess like this.
  4. Dave Honess

    Dave Honess
    Expand Collapse
    New Member

    • Requester
    Joined:
    Nov 14, 2017
    Messages:
    4
    Likes:
    11
    Ah I see. Thanks, yeah it's kind of hard to put into words concisely because they can use any phrase as text, which doesn't need to meet the dictionary definition of a greeting, or any 8x8 picture. There is a link to an example with an error under the "Entries must not" section of the collapsible instructions. Perhaps I should also have a link to it near the drop down for errors.

    I was also thinking of switching to radio buttons to make it less clicks for people.
     
    <Gucci> and turker like this.
  5. turker

    turker
    Expand Collapse
    How many pork chop of pork chop are in this image?

    • Contributor
    Joined:
    Jan 23, 2016
    Messages:
    11,789
    Likes:
    23,747
    I would have known for sure the right answer is yes if I had seen that
     
    Collapse Signature Expand Signature
    <Gucci> and Dave Honess like this.
  6. Dave Honess

    Dave Honess
    Expand Collapse
    New Member

    • Requester
    Joined:
    Nov 14, 2017
    Messages:
    4
    Likes:
    11
    Okay cool, thanks - this is helpful. I'll refine the instructions using this feedback for when I upload the big batch at the end of the month.
     
    <Gucci>, Evergreen Queen and turker like this.
  7. Dave Honess

    Dave Honess
    Expand Collapse
    New Member

    • Requester
    Joined:
    Nov 14, 2017
    Messages:
    4
    Likes:
    11
    Hi everyone, just to let you all know I've just published about 5000 of these HITs at 10c each (instructions updated slightly). Please feel free to grab a piece of the action. Many thanks!
     
    basketcasey and jan like this.
  8. Empress328

    Empress328
    Expand Collapse
    Semi Well-Known Member

    Joined:
    Feb 24, 2016
    Messages:
    643
    Likes:
    816
    Bah, this is only for masters.
     
    Cara likes this.
  9. Cara

    Cara
    Expand Collapse
    ad space now available

    • Contributor
    Joined:
    Jan 12, 2016
    Messages:
    4,822
    Likes:
    8,959
    Thank you for checking. I was just about to go look!
     
    Collapse Signature Expand Signature
    Empress328 likes this.
  10. electrolyte

    electrolyte
    Expand Collapse
    The Ghost of MTurk Past

    • Administrator
    Joined:
    Jan 10, 2016
    Messages:
    15,530
    Likes:
    36,564
    Hi! Is there a reason you're only allowing Masters workers? There are plenty of great workers on MTurk who haven't won the Masters lottery so they're not able to access HITs that have the Masters qualification on them.

    If you're trying to ensure high quality, you can set criteria that requires workers to have a high approval rating (maybe use "greater than 97% approval" or something similar).
     
    Collapse Signature Expand Signature
    Serial Mom likes this.