Rep Present Hummingbird Watch

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
Hello everyone!

We are working hard to have our HITs ready for the week long trial, and I'm happy to have a schedule update for you. The daily HIT schedule will go as follows (Pacific Time):
  1. 6:00 am
  2. 10:00 am
  3. 2:00 pm
  4. 4:30 pm
  5. 5:00 pm
  6. 5:30 pm
  7. 9:00 pm
The first day of the 7 day trial is this Friday (2016-08-12).

Each HIT will consist of 3 assignments, and should take less than a minute to complete.

More information to come! Thanks,

HW
Hi! Just to clarify, you will be posting 1 HIT with 3 assignments total at each time? So at 6:00am, 3 workers will be able to do the task, and so on for each time slot?
 
  • Like
Reactions: HummingbirdWatch

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
That's correct. For each specified time, three workers will be able to access the task. Thanks for asking!
 

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
Thanks for the response! If a worker accepts a task at 6:00am, can they also do the task at other time slots, or is it a one-per-person thing?
 

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
Thanks for the response! If a worker accepts a task at 6:00am, can they also do the task at other time slots, or is it a one-per-person thing?
A worker will be able to access a task at each time slot. In fact we're hoping to see a few repeat workers that can provide feedback during this week.
 

kayagee

New Member
Joined
Jun 22, 2016
Messages
41
Reaction score
76
Points
218
Age
37
Gender
Female
A worker will be able to access a task at each time slot. In fact we're hoping to see a few repeat workers that can provide feedback during this week.
I know this may be a silly question, but will your requester name on MTurk be HummingbirdWatch?
 

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
I'm looking forward to working with your hits. A requester that is putting in the work to get things right out of the gate and give feedback to develop their worker pool is a breath of fresh air. Good luck with this project.
Thanks Zacharias, we are excited for this trial and hope it leads to a more long term project very soon.
 

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
Hello everyone, here is some more information about our trial week HITs.

Recap: Trial starts tomorrow (Friday) and will go until Thursday.

HITs will be posted at the same time everyday, including the weekend at 6:00 am, 10:00 am, 2:00 pm, 4:30 pm, 5:00 pm, 5:30 pm, 9:00 pm.

Each HIT will consist of 3 assignments. This means that three workers will have access to the same HIT at each time slot. Workers can access as many time slots as they like.

Qualifications:

We decided not to create a custom qualification for this week, instead we are using MTurk’s qualifications:

1. HIT Approval Rate (%) for all Requesters' HITs greater than or equal to 95.

2. Number of HITs Approved greater than or equal to 10000.

3. Location is one of AU, CA, NZ, GB, US.

We will be monitoring the performance of repeat workers, and may decide to offer a custom Hummingbird Watch Qualification to those who submit quality work consistently.

Wage/Bonuses:

We are offering $0.14 per HIT, with a bonus of $0.01 to make a total of $0.15 per HIT. This approach allows us to provide feedback to workers. We will be allotting 2 minutes for the HIT, but it really shouldn’t take more than 1 minute.

What exactly will we be doing?

Inspecting 10-12 images in a column, then clicking the image/check-box if it contains the specified object.

Providing Feedback

We will be monitoring our threads at the following links, please feel free to offer feedback on our HITs:

http://turkernation.com/showthread.php?27594-Feedback-for-the-Requester&p=445469#post445469

http://www.mturkgrind.com/threads/hummingbird-watch-week-long-trial.29977/

http://www.mturkcrowd.com/threads/hummingbird-watch.1553/


We are very excited for this trial, and hope that it leads to a long term project! Good luck and have fun with our HITs :).


HW
 
Last edited:

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
Morning everyone, our first HIT has been published! It's called:
Hummingbird Watch Day 1 Batch 1

UPDATE:

Sorry everyone, there was an issue with payment. It is being processed now and will be uploaded ASAP.

UPDATE:

Payment issue resolved, the HIT is published! Thank you for your patience!
 
Last edited:

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
Hummingbird Watch Day 1 Batch 2 has been published! The HIT format has been updated: 5 assignments, 5 minutes and an example image are provided.
 

Squatty

.
Contributor
Crowd Pleaser
HIT Poster
Joined
Mar 17, 2016
Messages
21,286
Reaction score
44,989
Points
2,538
Age
35
Gender
Female
Hello, I just tried one of your Hits and found it quite enjoyable. The payment is very fair. The only complaint I would have is that the pictures are quite large. I have to scroll a lot on the screen in order to see them all.
 

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
Hello, I just tried one of your Hits and found it quite enjoyable. The payment is very fair. The only complaint I would have is that the pictures are quite large. I have to scroll a lot on the screen in order to see them all.
Glad you enjoyed it Squatty @Squatty. Thanks for the feedback, we're collecting all feedback during this trial week to improve our HIT delivery for the next test week.
 

humbleturker

Well-Known Member
Contributor
Crowd Pleaser
HIT Poster
Joined
Jul 8, 2016
Messages
10,554
Reaction score
23,278
Points
1,738
Age
33
Gender
Male
I got to get used to batch in this case being a series of test HITs. I keep trying to grab them but missing. :p
 

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
I got to get used to batch in this case being a series of test HITs. I keep trying to grab them but missing. :p
Thanks for your interest in our HITs humbleturker @humbleturker. There will be tests all week until Thursday night. After that we're going to do another test week, perhaps with a larger batch of HITs. If you're interested in receiving a Qualification to complete our HITs stay tuned, we want to have one ready for the next test week. We are interested in quality workers who are interested in accepting our hits regularly.
 

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
Hello, I just tried one of your Hits and found it quite enjoyable. The payment is very fair. The only complaint I would have is that the pictures are quite large. I have to scroll a lot on the screen in order to see them all.
I just tried a HIT, too, and had this same response. I also tried to click on some of the images to make them larger as the instructions suggested, but they didn't get larger. The page just jumped up to the top when I clicked on an image.

It would also be helpful to have a few links at the top to examples of what images with fire or smoke looks like so I have a better idea of what I'm looking for.

Otherwise, looks great! I'd enjoy doing a batch of these.
 
  • Like
Reactions: HummingbirdWatch

clickhappier

┬──┬ ノ( ゜-゜ノ)
Subforum Curator
Crowd Pleaser
Joined
Jan 12, 2016
Messages
728
Reaction score
1,634
Points
593
Location
USA
A worker will be able to access a task at each time slot. In fact we're hoping to see a few repeat workers that can provide feedback during this week.
Good luck with that since it's so hard to get even one of these. :( Between the unnecessarily tiny timer, different HIT group every time, and so few assignments each time... frustrating.

Finally caught one and the dang thing expired before I could get all the images to load. It doesn't matter how long the task 'should' take, I prefer HITs to have at least an hour available to reduce the chance of losing things to expirations if everything doesn't go perfectly. Never know when there'll be a personal interruption, internet slowdown, power outage, etc.
 
Last edited:
  • Like
Reactions: HummingbirdWatch

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
Good luck with that since it's so hard to get even one of these. :( Between tiny timer, different HIT group every time, and so few assignments each time... frustrating.
Thanks for the feedback clickhappier @clickhappier. Our project relies on very quick response time, within a few minutes from HIT creation to worker submission. This test week is on a small scale to test a couple of variables (speed, accuracy, getting to know the MTurk service, etc).

If we go forward with a long term project, it will be on a much larger scale with many more assignments than the current test. Also, there would be more HITs throughout the day (perhaps every hour or more).

Early results are promising, but more tweaking needs to be done on our part. If you have any ideas on how we could improve we'd be very thankful. For example, if there's a better delivery method than posting a different HIT group every time.

To recap, here's what we'd need for a longterm project with MTurk to be successful:

1. HITs to be accepted and submitted within a few minutes of being posted.
2. High accuracy for submissions.
3. HITs to be published every hour, sometimes more.

All feedback that has been provided will be included in a report that I present to my employer, and these changes will be applied to the next tests. We need to know how we can provide the mechanical turk workers with the tools to meet our needs and expectations. Keep the feedback coming, and see the future HITs become more worker friendly and enjoyable :).

HW

EDIT: I forgot to mention that we are interested in collecting a list of workers who are interested in becoming long term workers for our HITs. We will require a Qualification to work on our HITs in the future, and are interested in developing and training those who would like priority access to our HITs.
 
Last edited:

HummingbirdWatch

New Member
Requester
Joined
Aug 5, 2016
Messages
39
Reaction score
28
Points
18
I just tried a HIT, too, and had this same response. I also tried to click on some of the images to make them larger as the instructions suggested, but they didn't get larger. The page just jumped up to the top when I clicked on an image.

It would also be helpful to have a few links at the top to examples of what images with fire or smoke looks like so I have a better idea of what I'm looking for.

Otherwise, looks great! I'd enjoy doing a batch of these.
Thank you electrolyte @electrolyte. We overlooked the importance of having examples early on, and this will be one of the major changes we apply to the next test. We decided not to reject any submissions for this test to compensate for this oversight. Instead, we're accepting all submissions in the hopes that workers like yourself will feel encouraged to reach out to us and help us improve. Thanks for taking the time!

HW
 

Metallica

Never free, never me
Contributor
Joined
Jan 17, 2016
Messages
5,319
Reaction score
14,228
Points
1,113
Age
33
Gender
Male
never mind, I see my question was somewhat answered already.
 
Last edited: