Rep Present Feedback request for educational challenge entry verification

Dave Honess

New Member
Requester
Joined
Nov 14, 2017
Messages
4
Reaction score
11
Points
3
Gender
Male
Hi everyone!

I'm the programme manager for Astro Pi at the Raspberry Pi Foundation in Cambridge UK. Astro Pi is an annual science and coding competition where student-written code is run on the International Space Station to perform scientific experiments. It began as one of the educational projects for British ESA astronaut Tim Peake in 2015, open to the UK only, and was extended to all ESA member states for the flight of French ESA astronaut Thomas Pesquet last year. This year astronauts Paolo Nespoli and Alexander Gerst are the ambassadors.

I'm using MTurk to verify that the entries for one of our challenges, Mission Zero, have obeyed the rules.

I recently became a requester for the first time and uploaded 30 of these entries, one entry per HIT with 3 assignments each, as a test.

Without any further explanation I'd like to see how people find the instructions I've provided, is everything clear? When the challenge is over on the 26th, I'll be uploading probably around 1000 more of these.

s3.amazonaws.com/mturk_bulk/hits/291180703/rzWxRlzdUgVpYyZSm_eH0Q.html

Many thanks!

Dave
 
Last edited:

turker

How many pork chop of pork chop are in this image?
Contributor
Joined
Jan 23, 2016
Messages
11,804
Reaction score
23,770
Points
1,288
Hi everyone!

I'm the programme manager for Astro Pi at the Raspberry Pi Foundation in Cambridge UK. Astro Pi is an annual science and coding competition where student-written code is run on the International Space Station to perform scientific experiments. It began as one of the educational projects for British ESA astronaut Tim Peake in 2015, open to the UK only, and was extended to all ESA member states for the flight of French ESA astronaut Thomas Pesquet last year. This year astronauts Paolo Nespoli and Alexander Gerst are the ambassadors.

I'm using MTurk to verify that the entries for one of our challenges, Mission Zero, have obeyed the rules.

I recently became a requester for the first time and uploaded 30 of these entries, one entry per HIT with 3 assignments each, as a test.

Without any further explanation I'd like to see how people find the instructions I've provided, is everything clear? When the challenge is over on the 26th, I'll be uploading probably around 1000 more of these.

s3.amazonaws.com/mturk_bulk/hits/291180703/rzWxRlzdUgVpYyZSm_eH0Q.html

Many thanks!

Dave
greeting was a sideways mushroom followed by the words "good luck". so if I was doing those hits, I don't know if I should put yes or no for greeting. also I couldn't see the example of an error. these look easily doable if those 2 things are more clear.
 

kryss

carburante al fuoco
Former Admin
Joined
Jan 10, 2016
Messages
20,907
Reaction score
61,558
Points
1,488
Without any further explanation I'd like to see how people find the instructions I've provided, is everything clear? When the challenge is over on the 26th, I'll be uploading probably around 1000 more of these.
Instructions seem clear to me.

Thanks for reaching out!
 

Dave Honess

New Member
Requester
Joined
Nov 14, 2017
Messages
4
Reaction score
11
Points
3
Gender
Male
greeting was a sideways mushroom followed by the words "good luck". so if I was doing those hits, I don't know if I should put yes or no for greeting. also I couldn't see the example of an error. these look easily doable if those 2 things are more clear.
Ah I see. Thanks, yeah it's kind of hard to put into words concisely because they can use any phrase as text, which doesn't need to meet the dictionary definition of a greeting, or any 8x8 picture. There is a link to an example with an error under the "Entries must not" section of the collapsible instructions. Perhaps I should also have a link to it near the drop down for errors.

I was also thinking of switching to radio buttons to make it less clicks for people.
 
  • Like
Reactions: <Gucci> and turker

turker

How many pork chop of pork chop are in this image?
Contributor
Joined
Jan 23, 2016
Messages
11,804
Reaction score
23,770
Points
1,288
Ah I see. Thanks, yeah it's kind of hard to put into words concisely because they can use any phrase as text, which doesn't need to meet the dictionary definition of a greeting, or any 8x8 picture. There is a link to an example with an error under the "Entries must not" section of the collapsible instructions. Perhaps I should also have a link to it near the drop down for errors.

I was also thinking of switching to radio buttons to make it less clicks for people.
I would have known for sure the right answer is yes if I had seen that
 

Dave Honess

New Member
Requester
Joined
Nov 14, 2017
Messages
4
Reaction score
11
Points
3
Gender
Male
Hi everyone, just to let you all know I've just published about 5000 of these HITs at 10c each (instructions updated slightly). Please feel free to grab a piece of the action. Many thanks!
 
  • Like
Reactions: basketcasey and jan

Empress328

Semi Well-Known Member
Joined
Feb 24, 2016
Messages
877
Reaction score
1,338
Points
393
Location
NYC
Gender
Female
Bah, this is only for masters.
 
  • Like
Reactions: Cara

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,975
Points
1,313
Hi everyone, just to let you all know I've just published about 5000 of these HITs at 10c each (instructions updated slightly). Please feel free to grab a piece of the action. Many thanks!
Hi! Is there a reason you're only allowing Masters workers? There are plenty of great workers on MTurk who haven't won the Masters lottery so they're not able to access HITs that have the Masters qualification on them.

If you're trying to ensure high quality, you can set criteria that requires workers to have a high approval rating (maybe use "greater than 97% approval" or something similar).
 
  • Like
Reactions: Serial Mom