steelersfan 38 Posted December 9, 2014 Report Share Posted December 9, 2014 Hi, I am working on a seemingly simplistic bot that searches google and other search engines for a user input url based on a user input keyword, if the url is not found, the bot should keep going through the pages until it is, then list what page it is on. I had worked with my friend on something similar years ago, but the code won't work in ubot 5 now. so having been away so long from botting and having lost a lot of my knowledge on the subject, I need a bit of a refresher on how to accomplish this task. I will attach my bot here for reference, hopefully someone can tell me what I am doing wrong, and how to fix it. Thanks! FYI- the old project I worked on with Han linked below: http://www.ubotstudio.com/forum/index.php?/topic/6040-finding-your-page-in-google-automation-possible I tried to run all of the bots there to see, but none work in v5. site-finder-bot-003.ubot Quote Link to post Share on other sites
steelersfan 38 Posted December 10, 2014 Author Report Share Posted December 10, 2014 I solved the loop issue somewhat, although I need to test a little more and come up with the best solution for the rest of the functionality. Hopefully this helps other people looking to do similar things. I see a lot of stuff on here that uses scraping, but not much of this type of examples. And I am sure to run into further hurdles on the way. Honestly, I am quite proud of myself that I have remembered as much as I have! site-finder-bot-004.ubot Quote Link to post Share on other sites
steelersfan 38 Posted December 10, 2014 Author Report Share Posted December 10, 2014 I am having a bit of trouble with how deep the bot goes, if set to 10 pages it never actually goes that far, it seems bugged out. I am also trying to figure out how to make it report what page it is found on in the alert. Then I am going to tackle how to add the other search engines workings, which should be fun lol. Anyone have any insights or advice? Quote Link to post Share on other sites
Edward_2 85 Posted December 11, 2014 Report Share Posted December 11, 2014 Just use some regex to scrape the urls from G add list to list(%results, $find regular expression($scrape attribute(<class="r">, "innerhtml"), "(?<=href\\=\\\")http.*?(?=\\\")"), "Delete", "Global") I wouldn't bother searching past the 3rd or 4th page. Scrape urls, find the keyword in the list position, divide that by 10... voila. Quote Link to post Share on other sites
steelersfan 38 Posted December 11, 2014 Author Report Share Posted December 11, 2014 Thank you Edward, but I'm not sure what you mean? The whole idea of the bot is that it finds position, no matter how deep. It is for tracking the position when doing SEO for clients. It does not need a list or to save to, or pull from, as far as I can see. I want it to be single input and output, like it is now. The only purpose is for a user to input a keyword and their URL, then let it search and find what page they are on, no need for even the actual position on the page. It is really for internal use mostly, but will be given away for free as a promotional tool as well. I have it working, the only problem now is the looping won't work properly unless I set it to 200, and I need a way to report on the page that it stops at. Quote Link to post Share on other sites
lexnonscripta 0 Posted December 24, 2015 Report Share Posted December 24, 2015 Has this been solved? Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.