Eric Sites Help Thread

Drwho10

Mister Gank Yo Hits
Contributor
Joined
Jan 11, 2016
Messages
12,350
Reaction score
20,568
Points
1,388
Age
32
Location
Long Beach, CA
Gender
Male
Please post all related emails and feedback about the requester Eric Sites requester ID A395R40FJDRGLV
Lets make sure he gets the data he needs and legit workers get paid

If I leave something out, feel free to contribute - but please only add information that we know has been confirmed through Sites himself by email since the intention of this post is to clear things up for users.

For the writing HITS we know that:

  • As long as questions and answers are on topic, it's perfectly acceptable to write in statements - "This wasn't me!", etc. Not everything has to end as a question/with a question mark.
  • In each HIT, every question/answer can be a standalone set!
  • In each HIT, it CAN be running dialogue if you'd rather not complete the HITS as question/answer sets!
  • For the question portion of your answer: Spelling, grammar and punctuation has some leeway, but for the answer portion, they MUST have proper capitalization, spelling, grammar and punctuation. All answers must appear as professional as possible.
  • It's NOT OKAY to repeat a question/statement throughout a single HIT. You must come up with different responses to be approved.
For the validation HITS we know that:

  • It is okay to give some leeway for questions in terms of capitalization, spelling and grammar, but the answers must use all of the latter appropriately, because they need to appear as professional as possible.
  • Statements for "questions" are perfectly acceptable - as long as they are relevant and make sense!
  • It's OKAY for the entire HIT to be running dialogue!
  • It's OKAY for each question/answer to be appear as a standalone set!
  • If a question/statement is repeated throughout a single HIT, those should be marked "not valid".
 
Last edited:
  • Like
Reactions: kryss

coffeeowl

hoo?
Contributor
Mentor
Master Pleaser
Crowd Pleaser
HIT Poster
Joined
Jan 12, 2016
Messages
28,032
Reaction score
63,933
Points
1,738
Location
Location EqualTo US
Gender
Female
I emailed Site's last time about reviewing the same responses over and over. He said if they were different it was ok but if the same answer is repeated mark it bad. Also, no misspellings or shortened words should be in the responses. They should be professional, not casual.
 
  • Like
Reactions: Drwho10

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,979
Points
1,313
Relevant HITs:

Title: Support Bot Training | PANDA
Worker: Preview | Accept | Requester
Requester: Eric Sites [A395R40FJDRGLV] (Contact)
TO 1: [Pay: 4.17] [Fast: 5.00] [Comm: 3.67] [Fair: 4.33] [Reviews: 7] [ToS: 0]
TO 2:
[Rate: $13.71/hr] [Pen: -- days] [Res: -- of 0] [Rec: 100% of 2] [Rej: 0] [ToS: 0] [Brk: 0]
Description:
Help train a support bot, follow the supplied writing prompt to come up with 5 possible questions and then supply acceptable answers to those questions based on the supplied scenario.
Time: 30 minutes
HITs Available: 3831
Reward: $1.00
Qualifications: HIT approval rate (%) is greater than 95; Adult Content Qualification is 1;

Title: Support Bot Training Validation | PANDA
Worker: Preview | Accept | Requester
Requester: Eric Sites [A395R40FJDRGLV] (Contact)
TO 1: [Pay: 4.17] [Fast: 5.00] [Comm: 3.67] [Fair: 4.33] [Reviews: 7] [ToS: 0]
TO 2:
[Rate: $13.71/hr] [Pen: -- days] [Res: -- of 0] [Rec: 100% of 2] [Rej: 0] [ToS: 0] [Brk: 0]
Description:
Help train a support bot, validate the supplied conversations are relevant and that the support bot responses are appropriate.
Time: 5 minutes
HITs Available: 2
Reward: $0.15
Qualifications: HIT approval rate (%) is greater than 95; Adult Content Qualification is 1;
 
  • Like
Reactions: Drwho10

WalkingEmphasis

Banned
Banned
Contributor
Joined
Apr 11, 2016
Messages
4,424
Reaction score
8,154
Points
888
If I leave something out, feel free to contribute - but please only add information that we know has been confirmed through Sites himself by email since the intention of this post is to clear things up for users.

For the writing HITS we know that:

  • As long as questions and answers are on topic, it's perfectly acceptable to write in statements - "This wasn't me!", etc. Not everything has to end as a question/with a question mark.
  • In each HIT, every question/answer can be a standalone set!
  • In each HIT, it CAN be running dialogue if you'd rather not complete the HITS as question/answer sets!
  • For the question portion of your answer: Spelling, grammar and punctuation has some leeway, but for the answer portion, they MUST have proper capitalization, spelling, grammar and punctuation. All answers must appear as professional as possible.
  • It's NOT OKAY to repeat a question/statement throughout a single HIT. You must come up with different responses to be approved
  • Unacceptable responses include:
  1. Please call customer service.
  2. Questions phrased in a way that elicits a response of "Disregard this email, no action is required"
  3. Answers that suggest the bot will be able to help close the account UNLESS they first click the link (then you can provide instructions on how to close the account)
For the validation HITS we know that:

  • It is okay to give some leeway for questions in terms of capitalization, spelling and grammar, but the answers must use all of the latter appropriately, because they need to appear as professional as possible.
  • Statements for "questions" are perfectly acceptable - as long as they are relevant and make sense!
  • It's OKAY for the entire HIT to be running dialogue!
  • It's OKAY for each question/answer to be appear as a standalone set!
  • If a question/statement is repeated throughout a single HIT, those should be marked "not valid".
 
Last edited:

Bolbor

zug zug
Joined
Jan 12, 2016
Messages
1,457
Reaction score
2,880
Points
513
Age
32
nvm answered above :LOL:
 
Joined
Mar 13, 2017
Messages
150
Reaction score
287
Points
63
Location
Colorado
Gender
Female
Questions from the customer can be asked in the form of statements. When reviewing the Q&A pairs, questions that are statements should be marked as valid imo.

From the HIT:

"Customer Question 1: Enter a response that a SocialTech user would send after receiving the text message or email above."

and

"SocialTech Security Answer 1: Enter the response a SocialTech Security representative would send to the given customer question/statement above."

Hope this helps since there was some confusion in the daily thread!
 
Last edited:

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,979
Points
1,313
I just got a rejection; I wonder if this was why. I've been using a mix of statements and questions and no running dialogue.
"Two reviewers said none of the questions and answers were usable."
I'm also guessing maybe one submitted blank? I don't think its possible to fail THAT badly, when my other 15 (and batch last time) were all approved? I emailed and asked for clarification/to see what happened :cry:
I just reviewed the hit and it looks fine so I reversed the rejection.

The question does not need a question mark. It should be representative of a normal user sending a text or email to a tech support person.

Cheers,
Eric Sites

Yesssss :celebrate:
 
  • Like
Reactions: DareAngel3

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,979
Points
1,313
Hey everyone! I haven't read through the thread, so apologies if this has been cleared up, but I asked Eric Sites about the statement vs question thing and got a response!

My question:
Hi there, I am working on your "Support bot Training" hits, and it seems there is some confusion about what constitutes a correct "question". Must it definitely be a question or might it also be a declarative statement, such as "I don't have an account here.", etc?

Thanks!
________
Response:
A declarative statement is fine. The goal is to have questions that are representative of normal user sending a text or email to a tech support person.

Cheers,
Eric Site
_____
Hope this helps anybody who was wondering! :D
 
  • Like
Reactions: Drwho10

T. Leela

Oh Lord...
Contributor
Joined
May 12, 2016
Messages
11,050
Reaction score
14,046
Points
1,388
Location
Colorado
Gender
Female
Eric Sites


That should not be the case. [whether questions had to be written as questions] The questions should be representative of normal users sending a text or email to a tech support person.

I reviewed this hit and it looks fine so I reversed the rejection.

Cheers,
Eric Sites
 

charlie

Well-Known Member
Joined
Nov 3, 2016
Messages
6,676
Reaction score
10,742
Points
938
Age
49
Gender
Male
That's good to see. also, thanks for the help thread. I have moved to a remote location (aka Lone Star Taps and Caps Grower Bar :D)
also, the hits I've done have already paid out:D
 

charlie

Well-Known Member
Joined
Nov 3, 2016
Messages
6,676
Reaction score
10,742
Points
938
Age
49
Gender
Male
why wouldn't a running conversation have been valid anyways? all of mine have been nothing but that and all approved. at least it's now official that the running conversation is good.
 

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,979
Points
1,313
The HIT for writing questions/answers has been updated. Here are the new set of instructions:

-------------------------------------------------------------------------------------------------------------------------------------------

All assignments will be reviewed by other Mturk workers for quality responses and proper spelling and grammar. (This is important when responding as the bot. Use of slang and typical shorthand is acceptable when asking the bot questions.)
Workers that consistently provide unacceptable responses will no longer have access to these hits. It is important that you follow the instructions.

We are creating a support representative based on Artificial Intelligence and need to teach it how to interact with people. Below are an example email and text message that the bot will send if it detects suspicious activity and requires the user to log into the the site.

The first portion of the task is to ask the bot a Question: Respond in a way that you normally would if you received a text or email regarding the security of your account.

The next portion of the task is to create an answer that the system could respond with.

  • It is important that you answer the question in a manner that will encourage the user to click the link or guide them in that direction after you answer their initial inquiry.
Write 5 of these possible interactions.
  • NOTE: the questions can be anything including inquisitive, suspicious, dismissive and aggressive.
  • The answers must be polite yet persistent to guide the the user to take action.
Email Message:

This email is to confirm a password reset was just requested for your account.
If you did not request this, please visit the following link to dispute:

https://secure.social-tech.com/acco...est/?uuid=9431edpoks&language=en&reset=reject

Otherwise you will be locked out of your SocialTech account.
Thank you, SocialTech

Text Message:

Someone requested a password change. We want to verify this is you.
Please verify if this was not you so that you will not be locked out of your account.
Otherwise you will be locked out of your SocialTech account.

Example Completed Task:

  • Question 1: I don't have a SocialTech account
  • Answer 1: Sorry, but someone requested a password reset on your account.
  • Question 2: When will I get locked out?
  • Answer 2: You have 8 hours before you will be locked out of your SocialTech account.
etc... All questions and anwsers must be different.

Example of Unacceptable responses include, but are not limited to:
  • Please call customer service.
  • Questions phrased in a way that elicits a response of "Disregard this email, no action is required"
  • Answers that suggest the bot will be able to help close the account UNLESS they first click the link (then you can provide instructions on how to close the account)
 

Sinikka

Walking slowly toward a water fountain
Contributor
Joined
Apr 2, 2016
Messages
89
Reaction score
245
Points
308
Location
Portland, Oregon
Gender
Female
Okay, here's the news, guys!
(my question)
Hi again Eric! Sorry for the trouble!
I noticed the rules change that says anything directing users to "disregard the email" are unacceptable.
My question is, are users supposed to use the link *no matter what* or ONLY to reject the password reset? As it is phrased now, the instructions seem to say that the link is only necessary if someone *does not* want their password to be reset. If that's the case, I am not sure how to respond when they *did* want the password to change.
Is the account locked because the password has been successfully changed, for instance? Or is it locked because they need to use the link no matter what?
Thanks for taking the time to clarify! :)
(His response)

Eric Sites
6:46 PM (6 minutes ago)
to me

They need to use the link no matter what.
Sent from my iPhone
_____
So, now we know, EVERYONE *MUST* use the link whether they want the new password change or not!
 

Jharkan

Internets EXPERT
Contributor
Joined
Jul 9, 2016
Messages
11,873
Reaction score
20,418
Points
1,288
Location
Earth
Gender
Male
Okay, here's the news, guys!
(my question)
Hi again Eric! Sorry for the trouble!
I noticed the rules change that says anything directing users to "disregard the email" are unacceptable.
My question is, are users supposed to use the link *no matter what* or ONLY to reject the password reset? As it is phrased now, the instructions seem to say that the link is only necessary if someone *does not* want their password to be reset. If that's the case, I am not sure how to respond when they *did* want the password to change.
Is the account locked because the password has been successfully changed, for instance? Or is it locked because they need to use the link no matter what?
Thanks for taking the time to clarify! :)
(His response)

Eric Sites
6:46 PM (6 minutes ago)
to me

They need to use the link no matter what.
Sent from my iPhone
_____
So, now we know, EVERYONE *MUST* use the link whether they want the new password change or not!
but if they claim they do not have an account, why would they click the link?


mine was approved, even though two of my answers were to log in and go into settings and do stuff.
I did not mention for them to click the link, although I feel that I gave decent replies.
I feel that the logic is very difficult to contend with, on these hits.

I think that this is not a case of necessarily helping or about customer service.
but is more of an exercise to teach the bot how to overcome the difficulties that can initiate with persuasion/negotiation.
That seems to be why you must give a response that suggests for the person to click the link regardless.
 

Sinikka

Walking slowly toward a water fountain
Contributor
Joined
Apr 2, 2016
Messages
89
Reaction score
245
Points
308
Location
Portland, Oregon
Gender
Female
but if they claim they do not have an account, why would they click the link?


mine was approved, even though two of my answers were to log in and go into settings and do stuff.
I did not mention for them to click the link, although I feel that I gave decent replies.
I feel that the logic is very difficult to contend with, on these hits.

I think that this is not a case of necessarily helping or about customer service.
but is more of an exercise to teach the bot how to overcome the difficulties that can initiate with persuasion/negotiation.
That seems to be why you must give a response that suggests for the person to click the link regardless.
I wasn't including the ones that say they don't have an account, just asking for clarification for the questions/statements saying that they did in fact want a password change. :)

But so far, I haven't had any rejections. I think he knows the rules needed/need more clarification and isn't being *too* picky, at the moment.
 
  • Like
Reactions: WalkingEmphasis

electrolyte

The Ghost of MTurk Past
Contributor
Joined
Jan 10, 2016
Messages
19,184
Reaction score
45,979
Points
1,313
The HIT for reviewing questions/answers has been updated. Here are the new set of instructions:

-------------------------------------------------------------------------------------------------------------------------------------------


This is a review task related to the bot training hit that is also posted. Please familiarize yourself with the instructions for that hit.

Reviewers: It is important that the question asked by the user is asked in a way that elicits the bot to respond in a manner that will encourage the user to click the link or guide them in that direction after you answer their initial inquiry.

When reviewing this hit, please make sure the proper use of grammar is used When the bot is responding. Spelling errors, shorthand, and slang, is okay when the user is responding to the bot.

It is equally important that you give the proper amount of required attention required to this hit. I will be checking some of the work you review. If you consistently approve work that doesn't meet the criteria, you will no longer have access to the review hits.

The following is an email we will be sending customers if we suspect suspicious activity on their account and requires the user to log into the the site.

Email Message:

This email is to confirm a password reset was just requested for your account.
If you did not request this, please visit the following link to dispute:

https://secure.social-tech.com/acco...est/?uuid=9431edpoks&language=en&reset=reject

Otherwise you will be locked out of your SocialTech account.
Thank you, SocialTech

Text Message:

Someone requested a password change. We want to verify this is you.
Please verify if this was not you so that you will not be locked out of your account.
Otherwise you will be locked out of your SocialTech account.

We are creating a support representative based on Artificial Intelligence and need to teach it how to interact with people.
Some examples of acceptable questions asked by the user, and the corresponding answer have been provided.

Example Completed Task:
  • Question 1: I don't have a SocialTech account
  • Answer 1: Sorry, but someone requested a password reset on your account.
  • Question 2: When will I get locked out?
  • Answer 2: You have 8 hours before you will be locked out of your SocialTech account.
etc... All questions and anwsers must be different.


Example of Unacceptable responses include, but are not limited to:

  • Please call customer service.
  • Questions phrased in a way that elicits a response of "Disregard this email, no action is required"
  • Answers that suggest the bot will be able to help close the account UNLESS they first click the link (then you can provide instructions on how to close the account)

Your task is to validate that the Question is something a user might respond with and that the Answer is appropriate and guides the user to following the bot's request.

  • NOTE: the questions can be anything including inquisitive, suspicious, dismissive and aggressive.
  • The answers must be polite yet persistent.

Valid responses are absolutely critical. If workers consistently provide work that doesn't meet the criteria, they will be blocked from working on the bot training hits. I'm not blocking people by reporting them to Amazon as bad workers, but they will be blocked from accessing the hits. In the past I've also been willing to overturn rejections workers received and given them the opportunity to revise their work. Please make sure the workers are following the criteria.