Rep Present New Requester Veridium ID - HIT Posted

Veridium ID

New Member
Requester
Joined
Apr 3, 2018
Messages
7
Reaction score
0
Points
1
Age
43
Gender
Male
Veridium ID Data Collection - mturk project

Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Number of Assignments per HIT: 5
Reward per Assignment: $0.10
Number of HITS: 200

Average Time per Assignment: 35 seconds
 

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
Title: Data Collection using mobile phone and a custom application | AcceptA
Requester: Veridium ID [A22LGXWTD0S4ZK] Contact
TV: No Reviews
TO: No Reviews
TO2: No Reviews
Reward: $0.10
Duration: 5 minutes
Available: 200
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Requirements: None
 
  • Like
Reactions: Roscoe E Goldchain

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
Reading through your instructions, it says: "If you decide to quit before completing all 200 HITs, you will not receive any partial payments. In this case, the collected data will be discarded."

Does this mean if someone isn't able to get 200 HITs, then you will reject the HITs that they have already completed?
 

Veridium ID

New Member
Requester
Joined
Apr 3, 2018
Messages
7
Reaction score
0
Points
1
Age
43
Gender
Male
Reading through your instructions, it says: "If you decide to quit before completing all 200 HITs, you will not receive any partial payments. In this case, the collected data will be discarded."

Does this mean if someone isn't able to get 200 HITs, then you will reject the HITs that they have already completed?
If workers do more than 180 HITs, then they will get payed. The experiment needs a minimum number of HITs from a worker to be able to do the research project thus we put as mandatory request so that workers try to get to the end of the HITs.
 

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
If workers do more than 180 HITs, then they will get payed. The experiment needs a minimum number of HITs from a worker to be able to do the research project thus we put as mandatory request so that workers try to get to the end of the HITs.
Hmm, that could be a problem. Workers can't "reserve" 180 HITs for themselves, so it's quite likely that by the time one worker gets around to doing a large number, there could just not be that many HITs left to do because they've already been taken by other workers.
 

Veridium ID

New Member
Requester
Joined
Apr 3, 2018
Messages
7
Reaction score
0
Points
1
Age
43
Gender
Male
Hmm, that could be a problem. Workers can't "reserve" 180 HITs for themselves, so it's quite likely that by the time one worker gets around to doing a large number, there could just not be that many HITs left to do because they've already been taken by other workers.

We have developped a mechanism which counts the number of workers and also we track how many sessions are done for each worker. So, by adding their worker id, we reserve the 200 hits for that worker.
 

largeeyes

Well-Known Member
Joined
Jan 12, 2016
Messages
1,218
Reaction score
1,820
Points
413
Age
43
So, someone does 179 and then you give them 179 rejections? That can destroy a worker's account. I'm no engineer(actually, I am), but how can 180-200 be such a specifically significant sample size?
 

Veridium ID

New Member
Requester
Joined
Apr 3, 2018
Messages
7
Reaction score
0
Points
1
Age
43
Gender
Male
So, someone does 179 and then you give them 179 rejections? That can destroy a worker's account. I'm no engineer(actually, I am), but how can 180-200 be such a specifically significant sample size?
If there are 179 HITs we will pay the worker, it is an extreme example. We want to emphasize the fact that a worker should do all 200 to be relevant for us. From what we see as comments, I think the bonus approach is better.

The data sample acquired will be used for a machine learning flow and through the MTurk platform we gather more than the required samples, just to be sure that we reach the required sample size. The extra sample size is used for testing and also to simulate some real life scenarios.
 

largeeyes

Well-Known Member
Joined
Jan 12, 2016
Messages
1,218
Reaction score
1,820
Points
413
Age
43
If there are 179 HITs we will pay the worker, it is an extreme example. We want to emphasize the fact that a worker should do all 200 to be relevant for us. From what we see as comments, I think the bonus approach is better.

The data sample acquired will be used for a machine learning flow and through the MTurk platform we gather more than the required samples, just to be sure that we reach the required sample size. The extra sample size is used for testing and also to simulate some real life scenarios.
The bonus system is absolutely better. This way you can still pay if they don't complete them all, but also provide great pay if they do.
 

Veridium ID

New Member
Requester
Joined
Apr 3, 2018
Messages
7
Reaction score
0
Points
1
Age
43
Gender
Male
Title: Veridium ID Data Collection
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Qualification Requirement:

Number of Assignments per HIT: 1
Reward per Assignment: $15.00
Assignment duration: 2 hours
Approximate duration: 1 and 1/4 hours
Auto Approval Delay: 7 days

OBSERVATIONS:
1. Interface modified for a better user experience.
2. Hit mechanism modified - Submit button is available only after 200 Authentications, thus we reserve the worker the HIT he is doing.
3. Worker duplicate - by design, a worker can do only 200 Authentications. If he accepts another HIT, he will not be able to do another 200 Authentications. This way, we assure the uniqueness of the workers.
4. HIT Time - from our experiments, the HIT which consists in 200 Authentications, lasts approximately 1 hour and 15 minutes. We have set it 2 hours for convenience.
 

Veridium ID

New Member
Requester
Joined
Apr 3, 2018
Messages
7
Reaction score
0
Points
1
Age
43
Gender
Male
Title: Veridium ID Data Collection
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Qualification Requirement:

Number of Assignments per HIT: 1
Reward per Assignment: $18.00
Assignment duration: 3 hours
Approximate duration: 2
Auto Approval Delay: 7 days
 

A6_Foul_Out

I only use Organic TurkerView reviews
Contributor
HIT Poster
Joined
Aug 17, 2016
Messages
11,754
Reaction score
21,657
Points
1,538
Location
CT's Attic
Gender
Male
Title: Data Collection using mobile phone and a custom application | Accept
Requester: Veridium ID [A22LGXWTD0S4ZK] Contact
TV: No Reviews
TO: No Reviews
TO2: No Reviews
Reward: $18.00
Duration: 3 hours
Available: 1
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Requirements: None
[tr][td]
HIT exported from Mturk Suite v2.2.3
[/td][/tr]
[tr][td][/td][/tr]
 

DareAngel3

Well-Known Member
Contributor
Mentor
Crowd Pleaser
Joined
Jan 12, 2016
Messages
5,553
Reaction score
11,231
Points
2,713
Age
36
Location
Central IL
Gender
Female
Title: Veridium ID Data Collection
Description: Workers need to interact with Veridium ID application by using their mobile phones by doing an authentication, using mock up servers. During the interaction, data is collected using only mobile sensors e.g. accelerometer.
Keywords: data collection, mobile phone
Qualification Requirement:

Number of Assignments per HIT: 1
Reward per Assignment: $18.00
Assignment duration: 3 hours
Approximate duration: 2
Auto Approval Delay: 7 days
V @Veridium ID Hello! I just completed this HIT and had some polite constructive criticism. I tried contacting you via mturk, a response to the registration email, and via the email you provided in the HIT for support, but I never got a response.
My HIT started glitching about halfway through and would not progress. I finally figured out that I could restart the iframe/Mturk window to make it continue, but it continued to glitch throughout the experiment. I struggled to finish within the 3 hours. I would have been really frustrated if I could not submit after putting 2+ hours in, due to a glitch. The way the HIT is designed, I couldn't submit even if my timer ran out at 249/250 completed.
Also, the camera seemed to be completely random as to when it would fail or complete. Multiple times I took seemingly identical images, and some were accepted and others were not. I'm assuming this is what you're testing. However, you may consider increasing the timer so that it allots for both technical errors and "failed" captures, and gives enough time for workers to contact you and for you to respond.
Thanks, and I hope you get some really useful data!
 
  • Like
Reactions: electrolyte

Veridium ID

New Member
Requester
Joined
Apr 3, 2018
Messages
7
Reaction score
0
Points
1
Age
43
Gender
Male
V @Veridium ID Hello! I just completed this HIT and had some polite constructive criticism. I tried contacting you via mturk, a response to the registration email, and via the email you provided in the HIT for support, but I never got a response.
My HIT started glitching about halfway through and would not progress. I finally figured out that I could restart the iframe/Mturk window to make it continue, but it continued to glitch throughout the experiment. I struggled to finish within the 3 hours. I would have been really frustrated if I could not submit after putting 2+ hours in, due to a glitch. The way the HIT is designed, I couldn't submit even if my timer ran out at 249/250 completed.
Also, the camera seemed to be completely random as to when it would fail or complete. Multiple times I took seemingly identical images, and some were accepted and others were not. I'm assuming this is what you're testing. However, you may consider increasing the timer so that it allots for both technical errors and "failed" captures, and gives enough time for workers to contact you and for you to respond.
Thanks, and I hope you get some really useful data!

Hi DareAngel3,

Thank you for your feedback regarding the application and also for your effort for finishing the HIT. We have received other complaints for workers in this trial and we will start investigating the issues for the next run. This is the reason for the delay of a reply.

As a information for all workers, even if you finish only 150 Auth. from 250 Auth., you can mail us and we can reach an agreement of how to proceed further. You can get payed as per number of authentications done, until then, or receive a bonus compensation if you do the rest of the authentatications. In this scenario, we create a dedicated HIT using your WorkerId. We consider that every worker should be paid, especially when they really try to finish the HIT posted by us.

As per feedback, we will discuss with the development team and see what went wrong. From our validation process, we did not have a lot of problems, but it is a controlled environment versus real life environment. We hope that for our next trial all workers will have an improved user experience and a more stable application release.

The last request that we have, can you give us a private e-mail with the model of the phone?
 

DareAngel3

Well-Known Member
Contributor
Mentor
Crowd Pleaser
Joined
Jan 12, 2016
Messages
5,553
Reaction score
11,231
Points
2,713
Age
36
Location
Central IL
Gender
Female
Hi DareAngel3,

Thank you for your feedback regarding the application and also for your effort for finishing the HIT. We have received other complaints for workers in this trial and we will start investigating the issues for the next run. This is the reason for the delay of a reply.

As a information for all workers, even if you finish only 150 Auth. from 250 Auth., you can mail us and we can reach an agreement of how to proceed further. You can get payed as per number of authentications done, until then, or receive a bonus compensation if you do the rest of the authentatications. In this scenario, we create a dedicated HIT using your WorkerId. We consider that every worker should be paid, especially when they really try to finish the HIT posted by us.

As per feedback, we will discuss with the development team and see what went wrong. From our validation process, we did not have a lot of problems, but it is a controlled environment versus real life environment. We hope that for our next trial all workers will have an improved user experience and a more stable application release.

The last request that we have, can you give us a private e-mail with the model of the phone?
I sent a message to your account here on the forum with my model number. Thanks for your response!
 

Drex

New Member
Joined
Jan 12, 2016
Messages
4
Reaction score
0
Points
201
Age
44
Gender
Male
EDIT: for some reason it did not quote him
 
Last edited:

Drex

New Member
Joined
Jan 12, 2016
Messages
4
Reaction score
0
Points
201
Age
44
Gender
Male
Hi DareAngel3,

Thank you for your feedback regarding the application and also for your effort for finishing the HIT. We have received other complaints for workers in this trial and we will start investigating the issues for the next run. This is the reason for the delay of a reply.

As a information for all workers, even if you finish only 150 Auth. from 250 Auth., you can mail us and we can reach an agreement of how to proceed further. You can get payed as per number of authentications done, until then, or receive a bonus compensation if you do the rest of the authentatications. In this scenario, we create a dedicated HIT using your WorkerId. We consider that every worker should be paid, especially when they really try to finish the HIT posted by us.

As per feedback, we will discuss with the development team and see what went wrong. From our validation process, we did not have a lot of problems, but it is a controlled environment versus real life environment. We hope that for our next trial all workers will have an improved user experience and a more stable application release.

The last request that we have, can you give us a private e-mail with the model of the phone?
Where are we supposed to send the email to, because I sent a few yesterday and have yet to receive a response. I used the email that was listed in the hit. My problem was that it got to 202 sessions and would not advance past 202 no matter how many additional sessions that I completed. I sent an email as soon as I encountered the problem which occurred at about 2 hours and 10 minutes into the hit. I sent another a few minutes later with a screenshot of the problem and then sent one about 40 minutes later when the hit was about to expire. I had enough time to finish the hit, but it would not let me advance. I have no problem completing the additional sessions, but you have to understand how frustrating working on a hit for 2+ hours and it all of the sudden just stops working is for a worker and then to not get any responses to your emails and having no idea if you are going to be paid for your work is.
 
Last edited:

InfiniteChanges

Hiding In Your Ceiling
Contributor
Joined
May 1, 2017
Messages
3,426
Reaction score
8,150
Points
888
Gender
Male
Just wondering, will you be opening up HITs to workers who have already completed on of these tasks for you?
 

Drex

New Member
Joined
Jan 12, 2016
Messages
4
Reaction score
0
Points
201
Age
44
Gender
Male
I have currently sent 7 emails and have yet to receive a response. I sent three the day it happened including a screen shot, I sent one last Friday and another this past Monday all to the email listed in the hit. I have sent one today to a company email that another Turker who did get paid said they replied from and I sent another to the Turk address associated with the account. I did not expect replies to the ones that I sent today yet, but in case the requester posts tomorrow I want them to see this. I know that people got emails and hits setup yesterday for compensation, I was not one of them, I cant even get them to respond to an email.
 

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,978
Points
1,313
Tagging V @Veridium ID to come hopefully respond to the questions in this thread. (Perhaps they're not receiving notifications about thread replies.)
 
  • Like
Reactions: Drex