Concerning new "worker" site and Scraping Question

JohnH

New Member
Joined
Sep 8, 2017
Messages
12
Reaction score
12
Points
203
Age
56
Gender
Male
Apologies if you've read this somewhere else before...I'm posting this to as many places as I can find in hopes of an answer as time is running out.

Like much of everyone else, I'm trying to get my head wrapped around all of the new site's challenges. I typically do surveys.

My current strategy (with the old site):

First Tab I run MTurk with:
Pending Earnings
mmmTurkeyBacon Today's Projected Earnings
Panda Crazy Helper

Second Tab: Hit Scraper

Third Tab: Panda Crazy

Fourth Tab: Email (irrelevant to this topic...but hey, I like to be thorough)

The Problem: I've been poking around the new site for a few days as time permitted and have most of my issues worked out except for one big one....It is my understanding that Hit Scraper won't be usable on the new site. Okay, so I've been advised to transfer to using Hit Finder. So, I did. But, unless I'm missing something, there's one key component that really mucks things up for me. As far as I can tell, Hit Finder does just about everything the same as Hit Scraper did, but lacks a "Search Terms" field that would enable me to just be Scraping for "surveys". If I have to just search without this component, I'm going to have to visually comb through many more hits as they continuously pop up that aren't "surveys", which will slow me down very significantly.

By the Way...Overwatch grants me the ability to scrape by search term, but loses me the ability to also restrict the results to those above a certain payout...">.50" for example. So, that's not going to be the answer.

Panda Crazy also isn't the answer, as I'm not looking to auto accept.

I'm sort of surprised that this hasn't hit anyone else in the face yet, as I can't be the only one needing this.

J
 
  • Like
Reactions: <Gucci>

ecko

Well-Known Member
Contributor
Joined
Feb 2, 2016
Messages
3,764
Reaction score
6,813
Points
838
Age
41
Gender
Male
Apologies if you've read this somewhere else before...I'm posting this to as many places as I can find in hopes of an answer as time is running out.

Like much of everyone else, I'm trying to get my head wrapped around all of the new site's challenges. I typically do surveys.

My current strategy (with the old site):

First Tab I run MTurk with:
Pending Earnings
mmmTurkeyBacon Today's Projected Earnings
Panda Crazy Helper

Second Tab: Hit Scraper

Third Tab: Panda Crazy

Fourth Tab: Email (irrelevant to this topic...but hey, I like to be thorough)

The Problem: I've been poking around the new site for a few days as time permitted and have most of my issues worked out except for one big one....It is my understanding that Hit Scraper won't be usable on the new site. Okay, so I've been advised to transfer to using Hit Finder. So, I did. But, unless I'm missing something, there's one key component that really mucks things up for me. As far as I can tell, Hit Finder does just about everything the same as Hit Scraper did, but lacks a "Search Terms" field that would enable me to just be Scraping for "surveys". If I have to just search without this component, I'm going to have to visually comb through many more hits as they continuously pop up that aren't "surveys", which will slow me down very significantly.

By the Way...Overwatch grants me the ability to scrape by search term, but loses me the ability to also restrict the results to those above a certain payout...">.50" for example. So, that's not going to be the answer.

Panda Crazy also isn't the answer, as I'm not looking to auto accept.

I'm sort of surprised that this hasn't hit anyone else in the face yet, as I can't be the only one needing this.

J
people who want to only search for surveys with HF just limit the results to .50 and above. most batches pay less than this so it will filter them out. you can also set the TO rating to something like 3.0 so it will keep any bad ones from clogging up your results.
 

sryan66611

Well-Known Member
Contributor
Joined
Feb 13, 2016
Messages
3,412
Reaction score
5,893
Points
838
Age
36
Gender
Male
The Problem: I've been poking around the new site for a few days as time permitted and have most of my issues worked out except for one big one....It is my understanding that Hit Scraper won't be usable on the new site. Okay, so I've been advised to transfer to using Hit Finder. So, I did. But, unless I'm missing something, there's one key component that really mucks things up for me. As far as I can tell, Hit Finder does just about everything the same as Hit Scraper did, but lacks a "Search Terms" field that would enable me to just be Scraping for "surveys". If I have to just search without this component, I'm going to have to visually comb through many more hits as they continuously pop up that aren't "surveys", which will slow me down very significantly.

By the Way...Overwatch grants me the ability to scrape by search term, but loses me the ability to also restrict the results to those above a certain payout...">.50" for example. So, that's not going to be the answer.

Panda Crazy also isn't the answer, as I'm not looking to auto accept.

I'm sort of surprised that this hasn't hit anyone else in the face yet, as I can't be the only one needing this.

J
Yea.. So I've known about this for a while and as u said its interesting that no one brought it up. But as far as Hit scraper goes.. I think someone said the creator will be updating it for the new worker site afaik. I could be wrong but i am pretty sure I heard someone say this... As for that I have no answer about searching unless you are manually doing it as you said or the suggestion that Ecko gave you.


I will double check to see if its going to be updated and lyk JohnH @JohnH
 
  • Like
Reactions: <Gucci> and JohnH

JohnH

New Member
Joined
Sep 8, 2017
Messages
12
Reaction score
12
Points
203
Age
56
Gender
Male
Yea.. So I've known about this for a while and as u said its interesting that no one brought it up. But as far as Hit scraper goes.. I think someone said the creator will be updating it for the new worker site afaik. I could be wrong but i am pretty sure I heard someone say this... As for that I have no answer about searching unless you are manually doing it as you said or the suggestion that Ecko gave you.


I will double check to see if its going to be updated and lyk JohnH @JohnH
I have heard conflicting information as to whether Hit Scraper was going to be updated or not. At first it was, then the latest I heard it was a NO. I'd love a definitive statement.
 

JohnH

New Member
Joined
Sep 8, 2017
Messages
12
Reaction score
12
Points
203
Age
56
Gender
Male
people who want to only search for surveys with HF just limit the results to .50 and above. most batches pay less than this so it will filter them out. you can also set the TO rating to something like 3.0 so it will keep any bad ones from clogging up your results.
Trouble with that is I also want to pick up the qualifying hits for larger surveys that typically are way below .50 cents. So, that's not a good work around for me.
 

Jharkan

Internets EXPERT
Contributor
Joined
Jul 9, 2016
Messages
11,873
Reaction score
20,418
Points
1,288
Location
Earth
Gender
Male
what I did was went into the regular worker site, and did some searches that I would have liked to use hitscraper for
I then saved the resulting pages as bookmarks
Log in or register now. to view Spoiler content!

there are downsides
-inability to block requesters/titles
-inability to see [on the right side of hitscraper] if I've already done this task, or others by this requester [through lnk to hit database]
 

JohnH

New Member
Joined
Sep 8, 2017
Messages
12
Reaction score
12
Points
203
Age
56
Gender
Male
Well...I guess, since there doesn't seem to be a direct solution, I can just set up an "auto page reloader" on the main search. This, and everything else I've thought of or heard, seems like trying to run with crutches. At this time, this is as close as I can get to what's needed.

I guess at this point, my question starts to become "How hard would it be to get Hit Finder modified by someone to include a Search Terms field ?" This would in effect turn it into Hit Finder in Scraper's Clothing ?

This is such a simple, basic thing to need. I'm really at a loss as to why this hasn't been addressed. I mean...why wouldn't you put a Search Terms field in ? How useful would Google be, for example, if we could only search it by the "time that a page was crawled", or "who authored a page", or "whether we had previously liked it"; but not by "subject" or "contents" ?

The worker site almost does it by itself ! All that's missing there would be turning search results into a live feed, or including a user setting for auto search repeats.
 
  • Like
Reactions: <Gucci>

<Gucci>

Francis Manancis ...fuck you...
Threaderator
Crowd Pleaser
HIT Poster
Joined
Feb 6, 2016
Messages
13,614
Reaction score
29,767
Points
1,739
Age
38
Location
Detroit
Gender
Female
Well...I guess, since there doesn't seem to be a direct solution, I can just set up an "auto page reloader" on the main search. This, and everything else I've thought of or heard, seems like trying to run with crutches. At this time, this is as close as I can get to what's needed.

I guess at this point, my question starts to become "How hard would it be to get Hit Finder modified by someone to include a Search Terms field ?" This would in effect turn it into Hit Finder in Scraper's Clothing ?

This is such a simple, basic thing to need. I'm really at a loss as to why this hasn't been addressed. I mean...why wouldn't you put a Search Terms field in ? How useful would Google be, for example, if we could only search it by the "time that a page was crawled", or "who authored a page", or "whether we had previously liked it"; but not by "subject" or "contents" ?

The worker site almost does it by itself ! All that's missing there would be turning search results into a live feed, or including a user setting for auto search repeats.
Don’t be at a loss. This community is amazing. Maybe learn JavaScript and help some of these lovely people that write our scripts out.
Here is a thread where you can see what is currently being worked on.
http://www.mturkcrowd.com/threads/mturk-worker-userscripts.2716/
 

JohnH

New Member
Joined
Sep 8, 2017
Messages
12
Reaction score
12
Points
203
Age
56
Gender
Male
Hit Scraper has been converted as of today.