Advice on collecting pseudocode data

Sumith Kulal

New Member
Joined
Feb 6, 2019
Messages
2
Reaction score
3
Points
3
Age
23
Gender
Male
Hello everybody,

(new to the website here) I needed feedback from everybody for collecting pseudocode data. Our goal is generating simple English like sentences from lines of C++ code. A rough draft of the template is cs.stanford.edu/~sumith/mturk2.html

Note that the difficulty stems from C++ syntax, to aid with this we will:
1) Only use a small subset of C++ language with simple constructs.
2) Give plenty of hints so that the worker needn't know C++ beforehand.

The task might seem hard at first but after a couple of HITs should be very doable. We would want to hear your opinion on:
1) Is it doable? What can we do it to make better for the worker?
2) Is there any particular feedback for the interface?

Thanks in advance, looking forward to hearing from you.
Sumith
 
Last edited:

A6_Foul_Out

I only use Organic TurkerView reviews
Contributor
HIT Poster
Joined
Aug 17, 2016
Messages
11,752
Reaction score
21,654
Points
1,538
Age
25
Location
CT's Attic
Gender
Male
Note that the difficulty stems from C++ syntax
Based on the link, all of that looks like gibberish to me. So unless you specifically need workers without prior experience with C++ syntax, I would start by qualifying workers w/a qual hit who already have an understanding of that concept.

You can post a simple task that pays 0 and say:

I am looking for workers with this experience. This is the task. This is what I'm going to be paying. Prove to me your understanding C++ w/these questions. Workers who pass will be granted a qualification for hits in the future.

especially
if this is the case:
If something is unclear, we urge you to message us instead of filling a wrong answer
https://docs.aws.amazon.com/AWSMechTurk/latest/RequesterUI/AssigningaQualificationTypetoaWorker.html

For reference, another requester posted something similar as it pertains to converting html to English.

https://turkerview.com/search/hits/?q=Scott+Antipa

https://s3.amazonaws.com/mturk_bulk/hits/357943740/r8xF1iy2g_NsGMsBCHFRfg.html?assignmentId=3NLZY2D53PPO5UAM0Q8L7XNOYQILQX&hitId=307L9TDWJYSKZ99BOQ72JX3DDDYN3Z

Using this, as somebody with 0 experience, I was able to fully understand the concept. If you're hellbent on allowing all workers regardless of experience with C++ to participate, having a fully written out tutorial such as the one above would go a long way.
 
Last edited:
  • Like
Reactions: Sumith Kulal

Sumith Kulal

New Member
Joined
Feb 6, 2019
Messages
2
Reaction score
3
Points
3
Age
23
Gender
Male
Hi A6_Foul_Out @A6_Foul_Out ,

Thanks a lot for your response! Super helpful and the links are useful too! I'll start out with a qualification task and see if I get expected response :) and take it from there.

Sumith
 
  • Like
Reactions: <Gucci>

<Gucci>

Francis Manancis ...fuck you...
Threaderator
Crowd Pleaser
HIT Poster
Joined
Feb 6, 2016
Messages
13,610
Reaction score
29,747
Points
1,739
Age
35
Location
Detroit
Gender
Female
Hello everybody,

(new to the website here) I needed feedback from everybody for collecting pseudocode data. Our goal is generating simple English like sentences from lines of C++ code. A rough draft of the template is cs.stanford.edu/~sumith/mturk2.html

Note that the difficulty stems from C++ syntax, to aid with this we will:
1) Only use a small subset of C++ language with simple constructs.
2) Give plenty of hints so that the worker needn't know C++ beforehand.

The task might seem hard at first but after a couple of HITs should be very doable. We would want to hear your opinion on:
1) Is it doable? What can we do it to make better for the worker?
2) Is there any particular feedback for the interface?

Thanks in advance, looking forward to hearing from you.
Sumith
I think there are only a few turkers that can do this work. Count me in :gucci: