HIT Scraper Guide

kryss

carburante al fuoco
Former Admin
Joined
Jan 10, 2016
Messages
20,910
Reaction score
61,580
Points
1,488
HIT scraper is an invaluable tool for finding HITs. It's pretty simple to use, while providing the most efficient way to find HITs. You need to have Tampermonkey installed for chrome or Greasemonkey installed for firefox to install HIT scraper. Once you have tampermonkey or greasemonkey installed, go to this link and click on 'install this script'. Once you have HIT Scraper installed, you'll want to go to this link on mturk (It's a good idea to bookmark this link, as you'll probably be using it often):

Code:
https://www.mturk.com/mturk/findhits?match=false&hit_scraper

Once you access HIT Scraper on mturk, it will bring up HIT Scraper with the default settings.




You'll want to start by adjusting the settings.
  • Change 'Auto-refreshy delay" to your desired speed, such as 05 or 10 seconds. This settings is for how often it will scrape for new HITs.
  • Change 'Pages to scrape' to 1
  • Adjust 'Results per page' to something more reasonable then 10, such as 50. The maximum on this is 100, but it tends to cause a bit of lag if you have it set too high.
  • You can leave minimum reward at 0 or adjust it to .02c, 00.50, or you can even it set it 1.00, if you're only looking for higher paying surveys. Just keep in mind, any HITs that post below your threshold, will not show up on HIT Scraper.
  • You also have the option to only scrape for hits that you're qualified for.
  • The minimum batch size option is great if you're only looking to work on batches and stay away from surveys. You can set it to only show HITs that have at least 50 HITs in the batch. Make sure you tick the 'global' option if you're searching by latest.
  • The new HIT highlighting option is for how many seconds a HIT is highlighted once it initially pop ups on HIT scraper.
  • HIT scraper also has the option of pinging you when a new HIT comes in. This setting is really useful, as you don't have to be constantly watching HIT scraper.
  • You have the option to search between latest, most available, reward, and title. Setting your search at latest will enable you to see the newest HITs posting on mturk.
  • There's additional options for you to filter HITs via their TO ratings.

Once all your settings are adjusted, you're ready to click start. The columns on HIT scraper are pretty self explanatory. Take note, the panda links are under the 'Reward & PandA' column.


You can also tick 'Hide Panel' to collapse the settings while you're running HIT Scraper

Log in or register now. to view Spoiler content!













You also have the option to add a block list and an include list into scraper. You can add requesters to your block list by clicking the 'R' box. You can also block the title of the HIT by clicking the 'T' box in the requester column. Keep in mind, if you block the title of the HIT, it isn't going to block that particular requester. It will also block HITs from other requesters, if they have the same HIT title.



You can also achieve this by clicking on 'Edit Blocklist' and manually entering in requesters.

Log in or register now. to view Spoiler content!

To add requesters or titles to your include list, you need to click on 'Edit Includelist' and add them in.

Log in or register now. to view Spoiler content!



HIT scraper also has a forum exporter built into it. When you click on the 'vB' box, an export of the HIT will pop up. You can copy the text from the pop up and share it on the forum.




Tjololo @Tjololo, also made a great video guide for HIT Scraper.


If you have any questions, please feel free to ask.
 
Last edited:

WalkingEmphasis

Banned
Banned
Contributor
Joined
Apr 11, 2016
Messages
4,424
Reaction score
8,154
Points
888
I was wondering if you might be able to help me, or know someone who can?

I've been having an issue with the latest version of Hit Scraper. It won't allow me to change my background color when I run it in Chrome. After I go in to alter all of the colors, all of them change - except the background color. I'm running the latest version of Chrome and I'm using Windows 7. I have no idea what to do to fix it, but it's driving me crazy because I don't like having a white background. My sister is having the exact same issue. It works in Firefox, but I much prefer Chrome since all of the most useful scripts I'm using are running in Tampermonkey.

Any ideas?
 
  • Like
Reactions: Tribune

Tjololo

Active Member
Joined
Jan 12, 2016
Messages
629
Reaction score
2,628
Points
493
Age
35
I was wondering if you might be able to help me, or know someone who can?

I've been having an issue with the latest version of Hit Scraper. It won't allow me to change my background color when I run it in Chrome. After I go in to alter all of the colors, all of them change - except the background color. I'm running the latest version of Chrome and I'm using Windows 7. I have no idea what to do to fix it, but it's driving me crazy because I don't like having a white background. My sister is having the exact same issue. It works in Firefox, but I much prefer Chrome since all of the most useful scripts I'm using are running in Tampermonkey.

Any ideas?
Works for me, change the color and refresh. The background changes...well, except for the settings for some reason https://gyazo.com/dd98717044976dbb308ef0a662f3955a

EDIT:

Oh, ok, I'm just dumb. Control panel background is different than page background.
 
Last edited:

WalkingEmphasis

Banned
Banned
Contributor
Joined
Apr 11, 2016
Messages
4,424
Reaction score
8,154
Points
888
Works for me, change the color and refresh. The background changes...well, except for the settings for some reason https://gyazo.com/dd98717044976dbb308ef0a662f3955a

EDIT:

Oh, ok, I'm just dumb. Control panel background is different than page background.
Yeah, I can't figure it out. It's so frustrating. Everything *but* the background color changes. I don't get it.
 
  • Like
Reactions: Tribune

Tjololo

Active Member
Joined
Jan 12, 2016
Messages
629
Reaction score
2,628
Points
493
Age
35
Nope, I don't have anything like that.
The extensions question came from the author of the script, feihtality. He says that's the only reason why the background color wouldn't change. You must have something going on that's stopping it.

Try running it in incognito mode and changing the colors there. If it works there, disable all your extensions and scripts except that one, change the colors, then enable one by one until it breaks.
 

WalkingEmphasis

Banned
Banned
Contributor
Joined
Apr 11, 2016
Messages
4,424
Reaction score
8,154
Points
888
The extensions question came from the author of the script, feihtality. He says that's the only reason why the background color wouldn't change. You must have something going on that's stopping it.

Try running it in incognito mode and changing the colors there. If it works there, disable all your extensions and scripts except that one, change the colors, then enable one by one until it breaks.
Okay, thank you! I'll try that right now and provide an update when I'm done.
 
  • Like
Reactions: Tribune

WalkingEmphasis

Banned
Banned
Contributor
Joined
Apr 11, 2016
Messages
4,424
Reaction score
8,154
Points
888
Okay, I figured it out - it's Turk Assist. Kind of frustrating since I really like it, but I'll just have TA and HS open in different windows, one in an incognito. Unless there's some other kind of fix that people who are more knowledgeable than I am know about?

Regardless, thank you guys! I really appreciate the help, the white background was genuinely driving me crazy!
 
  • Like
Reactions: Tribune and kryss

Tjololo

Active Member
Joined
Jan 12, 2016
Messages
629
Reaction score
2,628
Points
493
Age
35
Okay, I figured it out - it's Turk Assist. Kind of frustrating since I really like it, but I'll just have TA and HS open in different windows, one in an incognito. Unless there's some other kind of fix that people who are more knowledgeable than I am know about?

Regardless, thank you guys! I really appreciate the help, the white background was genuinely driving me crazy!
Thanks for letting us know. How did you find out it was TA?
 
  • Like
Reactions: Tribune

WalkingEmphasis

Banned
Banned
Contributor
Joined
Apr 11, 2016
Messages
4,424
Reaction score
8,154
Points
888
Thanks for letting us know. How did you find out it was TA?
I had to open everything in an incognito window and disable the scripts one by one, as suggested, to see when HS would start working with each one disabled. For some reason (I don't know enough about programming to know why), doing that in a regular window didn't work. So now, I just have HS open in Firefox and I do the work in Chrome. Slightly annoying, but it works! :)
 

Kisha

New Member
Joined
May 15, 2020
Messages
1
Reaction score
0
Points
1
Age
49
Gender
Female
I've been having challenges with Hit Scrapper. when I select desired HITs, it opens a new Tab with either an Error message or Hits unavailable.
I've not successfully been able to use the Tool.
please help
 

Random Jobber

Well-Known Member
Joined
Feb 3, 2019
Messages
1,028
Reaction score
1,313
Points
413
Gender
Male
I've been having challenges with Hit Scrapper. when I select desired HITs, it opens a new Tab with either an Error message or Hits unavailable.
I've not successfully been able to use the Tool.
please help
HIT Scraper is outdated and doesn't receive updates anymore, you should use HIT Forker or the HIT Finder on MTurk Suite instead.
 
  • Like
Reactions: jan

RomeTurk

New Member
Joined
Feb 28, 2021
Messages
1
Reaction score
0
Points
1
Age
57
Gender
Male
You've brightened my Sunday, I hadn't used any additional tools in MTurk other than the original after two years of startup. I am eager to see the results of this tool. Thank you.
 

ChristopherASA

Well-Known Member
Contributor
Joined
Feb 18, 2016
Messages
11,716
Reaction score
23,143
Points
1,238
Age
76
Location
Wish it were Seattle,but it isn't. :(
Gender
Male
I've been out of this for SO LONG, I've forgotten basic stuff. Like what programs do I have to preload to make the word easier, more efficient, and ???? Greasemonkey, Hit Scraper, yada yada........... I'm at a complete loss. :(

Is there a site that will do a good job of re-explaining this stuff to me?

And sorry to put this load on the shoulders of those who try and help. :(