- Joined
- Apr 27, 2018
- Messages
- 3
- Reaction score
- 3
- Points
- 3
- Age
- 42
- Gender
- Male
Hi everyone,
I'm a newbie requester looking to get some feedback on a series of HITs for identifying and labeling forum/blog/Q&A sites. Early prototypes of the HITs (on the worker sandbox) are up at:
HIT #1: Identify forum sites:
https://workersandbox.mturk.com/projects/3GT259VZ8HNMZVGLGI8ZNNUORCVUY6/tasks
This is a simple single question with instructions. I've got keyboard shortcuts for the options for easy completion. Would it be helpful to have the site itself in an iframe so it displays inline? Or is it better as a link that opens in a new tab/window?
HIT #2: Find forum root:
https://workersandbox.mturk.com/projects/3YBH5XUK848XZQQBBMMX49NFP58UX5/tasks
This is a text box entry, again after looking at and manipulating a website. Same question about iframe vs. link as before. Also, is it clear enough what I mean by forum "root"? Would some examples help?
HIT #3-5: Label index/thread/post pages:
https://workersandbox.mturk.com/projects/3EHDCT959WNPFF90D07VRH2NT5YV02/tasks
https://workersandbox.mturk.com/projects/3YE2HAAZLWCM4LQQC4AI1U3QX6ZUZX/tasks
https://workersandbox.mturk.com/projects/38NH2CI7SPV5V1LGB7LFGOQU9VUVUZ/tasks
These involve a custom Javascript overlay that walks you through labeling several (~20) fields on a webpage. I'm most concerned about the granularity of these HITs - I know that it's best to break HITs up into individual tasks that can be completed quickly, but in these cases, the later stages are dependent upon what's selected in early stages, and I can use software to reduce the amount of work the Turker has to do or improve the error-checking on their input. (For example, if no categories are selected, it should hide the request to label category fields, and the set of legal click targets for the field is modified by the items selected in earlier stages.) Are these too daunting & long? Should I break them up so that each "stage" is a separate HIT, moving some of the software logic that's currently in the Javascript to the server? So that each individual labeling task is a separate HIT? Or is it more convenient to be able to click through a number of fields at once all on the same document?
I'm also interested in feedback on wording, instruction clarity, UI, and pricing, if you have any opinions on those. I'll likely change the color scheme to match the default templates and add some example screenshots in the future.
Thanks for any input!
I'm a newbie requester looking to get some feedback on a series of HITs for identifying and labeling forum/blog/Q&A sites. Early prototypes of the HITs (on the worker sandbox) are up at:
HIT #1: Identify forum sites:
https://workersandbox.mturk.com/projects/3GT259VZ8HNMZVGLGI8ZNNUORCVUY6/tasks
This is a simple single question with instructions. I've got keyboard shortcuts for the options for easy completion. Would it be helpful to have the site itself in an iframe so it displays inline? Or is it better as a link that opens in a new tab/window?
HIT #2: Find forum root:
https://workersandbox.mturk.com/projects/3YBH5XUK848XZQQBBMMX49NFP58UX5/tasks
This is a text box entry, again after looking at and manipulating a website. Same question about iframe vs. link as before. Also, is it clear enough what I mean by forum "root"? Would some examples help?
HIT #3-5: Label index/thread/post pages:
https://workersandbox.mturk.com/projects/3EHDCT959WNPFF90D07VRH2NT5YV02/tasks
https://workersandbox.mturk.com/projects/3YE2HAAZLWCM4LQQC4AI1U3QX6ZUZX/tasks
https://workersandbox.mturk.com/projects/38NH2CI7SPV5V1LGB7LFGOQU9VUVUZ/tasks
These involve a custom Javascript overlay that walks you through labeling several (~20) fields on a webpage. I'm most concerned about the granularity of these HITs - I know that it's best to break HITs up into individual tasks that can be completed quickly, but in these cases, the later stages are dependent upon what's selected in early stages, and I can use software to reduce the amount of work the Turker has to do or improve the error-checking on their input. (For example, if no categories are selected, it should hide the request to label category fields, and the set of legal click targets for the field is modified by the items selected in earlier stages.) Are these too daunting & long? Should I break them up so that each "stage" is a separate HIT, moving some of the software logic that's currently in the Javascript to the server? So that each individual labeling task is a separate HIT? Or is it more convenient to be able to click through a number of fields at once all on the same document?
I'm also interested in feedback on wording, instruction clarity, UI, and pricing, if you have any opinions on those. I'll likely change the color scheme to match the default templates and add some example screenshots in the future.
Thanks for any input!