giganut 109 Posted July 30, 2014 Report Share Posted July 30, 2014 Tested on: Windows 8.1, (RAM) 8.00 64-bit OS x64-based processor, HP Desktop, .NET Framework 4.5Tested In the main Browser and not in a Threading Environment.Hello everyone, I seam to have found a way that helps ubot consume less memory. I'm not sure if this is a method that is unique to my situation, or the sites I'm working with. This method is working for me on Windows 8.1, .NET Framework 4.5 as of right now. So I though I would share my findings so others can hopefully benefit from this method, and possibly find out if this is a universal fix for memory consumption across multiple versions of windows or not. If you can test this method I would be very happy to here your findings.So lets jump right in. Unfortunately I can't post the site I'm working with for obvious reasons, but I can tell you this. The site I'm Woking with has lots of images, Goggle advertising, and video ads as well. These property's of the site where making the page loads extremely heavy on ubot's memory. Every time the bot would navigate to a new page the memory would get bigger and bigger until the bot just stopped or the dreaded Browser.exe crash would occur.So this is what I found (On Accident) to fix the problem. Right before the bot try's to navigate to a new page I would use the change attribute command to remove the advertising and image containers from the page. Then I would run the "load html" command (Loading a blank page) before the navigate and wait 2 seconds before the actual navigate command ran.By doing this the memory of the main Browser.exe would start at 58MB and drop to 7.5MB once the blank page //asset//assets// was loaded, then once the navigate commad would run and the new page was fully loaded the Browser.exe memory would settle at 35MB in the main browser. Below you will find some example code of the method, a example bot and video.P.S I would love to here your feed back and reports about this method and if it's working for everyone in threads or not. Video: http://screencast.com/t/lo9rLBoJa Example Bot and source code: https://www.mediafire.com/?gntgl7ojcd3f30p Example Source Code: clear list(%images) navigate("http://imgur.com/", "Wait") wait for element(<class="logo-icon">, 10, "Appear") add list to list(%images, $scrape attribute(<a,class="image-list-link">, "fullhref"), "Delete", "Global") set(#imagespos, 0, "Global") set list position(%images, #imagespos) divider loop($list total(%images)) { change attribute($element offset(<class="panel">, 1), "innertext", "") change attribute(<id="side-gallery">, "innertext", "") change attribute(<class="panel advertisement">, "innertext", "") wait(2) load html("<head> </head> <body style=\"background-color:black;> <h1 style=\"font-family:verdana; \"><font size=\"6\" color=\"red\"><center><strong></strong></center></font></h1> </body> <footer> </footer>") wait(2) navigate($list item(%images, #imagespos), "Wait") wait for element($element offset(<class="panel">, 1), 10, "Appear") wait(2) increment(#imagespos) } 1 Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted July 30, 2014 Report Share Posted July 30, 2014 You can run the load html node by itself first, wait 1.5 and then navigate this will cause the cache to clear itself. 1 Quote Link to post Share on other sites
giganut 109 Posted July 30, 2014 Author Report Share Posted July 30, 2014 You can run the load html node by itself first, wait 1.5 and then navigate this will cause the cache to clear itself. Thanks Tj, that will surly help to reduce the code in some cases. I would like to add that in the bot I was having problems with, I was seeing the main Browser.exe consumeas much a 1000MB before implementing this method. Once this method was implemented the memory consumption was stabilized at a steady pace of 58MB to 78MB Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted July 31, 2014 Report Share Posted July 31, 2014 good to know stuff thanks guys 2 Quote Link to post Share on other sites
giganut 109 Posted July 31, 2014 Author Report Share Posted July 31, 2014 good to know stuff thanks guys Thanks TC, did you get a chance to give this a test run by chance? Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted July 31, 2014 Report Share Posted July 31, 2014 Yeah bro, I tested in same system almost W7 laptop.Works as stated above....TJ's too I have a tendency to use "close page" in my threaded browser bots ...the few I have left Also for the kids watching at home, instead of a wait 2 seconds I use wait for element to DISAPPEAR, saves time!! like sooo... clear list(%images) navigate("http://imgur.com/", "Wait") wait for element(<class="logo-icon">, 10, "Appear") add list to list(%images, $scrape attribute(<a,class="image-list-link">, "fullhref"), "Delete", "Global") set(#imagespos, 0, "Global") set list position(%images, #imagespos) divider loop($list total(%images)) { change attribute($element offset(<class="panel">, 1), "innertext", "") change attribute(<id="side-gallery">, "innertext", "") change attribute(<class="panel advertisement">, "innertext", "") wait(2) close page navigate($list item(%images, #imagespos), "Wait") wait for element($element offset(<class="panel">, 1), 10, "Appear") wait for element($element offset(<class="panel">, 1), 2, "Disappear") increment(#imagespos) } stop script wait(2) wait(2) load html("<head> </head> <body style=\"background-color:black;> <h1 style=\"font-family:verdana; \"><font size=\"6\" color=\"red\"><center><strong></strong></center></font></h1> </body> <footer> </footer>") Like in your example using the load html then wait 2 seconds, wait for it to disapear is forgotten I think. I used to forget it all the time. I just wanted to remind the new kids it's there. I have also used navigate to bogus url with the close page for thread hanging. Someone shared while ago some where in the forum. TC Quote Link to post Share on other sites
giganut 109 Posted July 31, 2014 Author Report Share Posted July 31, 2014 Yeah bro, I tested in same system almost W7 laptop.Works as stated above....TJ's too I have a tendency to use "close page" in my threaded browser bots ...the few I have left Also for the kids watching at home, instead of a wait 2 seconds I use wait for element to DISAPPEAR, saves time!! like sooo... Like in your example using the load html then wait 2 seconds, wait for it to disapear is forgotten I think. I used to forget it all the time. I just wanted to remind the new kids it's there. I have also used navigate to bogus url with the close page for thread hanging. Someone shared while ago some where in the forum. TC That is absolutely awesome TC, I'm really glad to here this method is working for other users on different platforms as well as mine. Thanks for the report. I hope to here from other members as well about this method. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.