Press/Studies Attention Checks Reconsidered

[Achilles2357]

This is from last year, but I just discovered it today:
https://www.qualtrics.com/blog/using-attention-checks-in-your-surveys-may-harm-data-quality/

I find the reasoning fairly convincing.

It looks like there is some other interesting stuff on the qualtrics site too, in the blog and in the guidelines for researchers.

(I hope I'm posting in the right sub-forum. I guess I consider qualtrics part of the "mturk experience".)

 

[Chloe Cannaday]

I am new to the scene here, but that was a very interesting article.

I have found personally, that I tend to react to "attention checks" as rude, and when personality questions are asked both before and after such an occurrence, my reactions tend to skew into the realm of "pissed off".

 

[Achilles2357]

I am new to the scene here, but that was a very interesting article.

I have found personally, that I tend to react to "attention checks" as rude, and when personality questions are asked both before and after such an occurrence, my reactions tend to skew into the realm of "pissed off".

I've done this for around a year now, and if they have any one of the "normal" attention checks, I just roll with it. But if someone tries to get "clever" and throw in an attention check in some strange way, this really throws me off. Part of getting better at mturk is learning what sort of stuff you can just click through with no thinking, so that you can actually pay attention when it matters, but if they are trying to screw around, then this screws it all up.

 

[Ayacalypso]

On another format I use, the attention checks frequently have dummy responses. If you chose one, they know you didn't read it, or you are lying. Questions like "In the past 12 months have you purchased...?" and the responses will include normal stuff like, a car, refrigerator, diet soda, toaster pastries, but them obviously wrong responses such as "automatic table washer," or my favorite - "insect mucilage."

 

[Achilles2357]

On another format I use, the attention checks frequently have dummy responses. If you chose one, they know you didn't read it, or you are lying. Questions like "In the past 12 months have you purchased...?" and the responses will include normal stuff like, a car, refrigerator, diet soda, toaster pastries, but them obviously wrong responses such as "automatic table washer," or my favorite - "insect mucilage."

I did a survey on some cheesy site recently, and one of the options for my hobbies was "dinosaur trainer".