Welcome to MTurk Crowd

Work on Amazon Mechanical Turk, learn from the best, and have fun doing it. Join the crowd today!

Sign Up

Advice on collecting pseudocode data

Discussion in 'Requesters' started by Sumith Kulal, Feb 6, 2019.

  1. Sumith Kulal

    Sumith Kulal
    Expand Collapse
    New Member

    Joined:
    Feb 6, 2019
    Messages:
    2
    Likes:
    3
    Hello everybody,

    (new to the website here) I needed feedback from everybody for collecting pseudocode data. Our goal is generating simple English like sentences from lines of C++ code. A rough draft of the template is cs.stanford.edu/~sumith/mturk2.html

    Note that the difficulty stems from C++ syntax, to aid with this we will:
    1) Only use a small subset of C++ language with simple constructs.
    2) Give plenty of hints so that the worker needn't know C++ beforehand.

    The task might seem hard at first but after a couple of HITs should be very doable. We would want to hear your opinion on:
    1) Is it doable? What can we do it to make better for the worker?
    2) Is there any particular feedback for the interface?

    Thanks in advance, looking forward to hearing from you.
    Sumith
     
    #1 Sumith Kulal, Feb 6, 2019
    Last edited: Feb 6, 2019
    <Gucci> and A6_Foul_Out like this.
  2. A6_Foul_Out

    A6_Foul_Out
    Expand Collapse
    I only use Organic TurkerView reviews

    • HIT Poster
    • Contributor
    Joined:
    Aug 17, 2016
    Messages:
    11,481
    Likes:
    21,413
    Based on the link, all of that looks like gibberish to me. So unless you specifically need workers without prior experience with C++ syntax, I would start by qualifying workers w/a qual hit who already have an understanding of that concept.

    You can post a simple task that pays 0 and say:

    I am looking for workers with this experience. This is the task. This is what I'm going to be paying. Prove to me your understanding C++ w/these questions. Workers who pass will be granted a qualification for hits in the future.

    especially
    if this is the case:
    https://docs.aws.amazon.com/AWSMechTurk/latest/RequesterUI/AssigningaQualificationTypetoaWorker.html

    For reference, another requester posted something similar as it pertains to converting html to English.

    https://turkerview.com/search/hits/?q=Scott+Antipa

    https://s3.amazonaws.com/mturk_bulk...NOYQILQX&hitId=307L9TDWJYSKZ99BOQ72JX3DDDYN3Z

    Using this, as somebody with 0 experience, I was able to fully understand the concept. If you're hellbent on allowing all workers regardless of experience with C++ to participate, having a fully written out tutorial such as the one above would go a long way.
     
    Collapse Signature Expand Signature
    #2 A6_Foul_Out, Feb 6, 2019
    Last edited: Feb 6, 2019
    Sumith Kulal likes this.
  3. Sumith Kulal

    Sumith Kulal
    Expand Collapse
    New Member

    Joined:
    Feb 6, 2019
    Messages:
    2
    Likes:
    3
    Hi @A6_Foul_Out@A6_Foul_Out ,

    Thanks a lot for your response! Super helpful and the links are useful too! I'll start out with a qualification task and see if I get expected response :) and take it from there.

    Sumith
     
    <Gucci> likes this.
  4. <Gucci>

    <Gucci>
    Expand Collapse
    Francis Manancis ...fuck you...

    • Threaderator
    • Crowd Pleaser
    • HIT Poster
    • Mentor
    Joined:
    Feb 6, 2016
    Messages:
    13,577
    Likes:
    29,575
    I think there are only a few turkers that can do this work. Count me in :gucci:
     
    Collapse Signature Expand Signature