Best practices ,caveats and code for working with Ubot Browser

[Sanjeev 46]

Guys n Gal?

 

I have tried to collect here what I found relevant and important with regard to working with the browser by searching this forum-

Most of the credit for this goes to UbotDev and TJ. If I have missed crediting someone please let me know I'll add them in.

Please contribute to this post by sharing your knowledge and correcting my post as you see fit. i would like this to be central place to store all info related to working with the browser.

 

Thanks.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Current issues with the Ubot browser -

-computer memory usage  goes up as the bot runs, so each time it visits a new web page it uses up a new piece of memory.
-navigating to many webpages - over time will use up the pc's memory.

-many time the browser.exe process will stay active after bot is done.
-In the background Ubot will open hundreds of 'browser.exe' processes, which will cause the computer to slow down and the bot to freeze.
-If the bot uses the same instance of the browser for visiting/navigating to many web pages  - then after a certain amount of time it will bog
down the computer...
-You can use the same instance of the browser to navigate to multiple web pages in a row, but you need to re-initialize the browser.

-Currently 'out of the box' there is no  command/node  for completely clearing - the cache and all cookies (i.e including flash cookies).

the correct way to perform web page based tasks would be to -
 -open a new browser instance
 -preform the task
 -then close that browser instance

In practice -
use the "set user agent" command to re-initialize the 'main browser'.
use "in new browser" command -  this command initializes a new browser.
use "close page" command, to close the "browser.exe" process. - (Sometimes using 'in new browser' leaves the process "browser.exe" running, so you might use "close page" command at the end of your process, so that process browser.exe terminates.)
set user agent -  would also apply to "in new browser" command if you have it opened for a longer time (you can re-initialize before every
navigate command).

 

Browser.exe crash workarounds-

Also there is a plugin 'File Folder Commands' again from botguru.net -which will let you monitor browser.exe processes and this can be used to terminate the browser.exe instances if and when they crash.

- I wish we had some sample code to play with though.

- Update -This free bot provided generously by a fellow Ubotter  seems to be be a good wayr for cleaning up all the dead browsers from memory -

  Download it  here

 

The code below is From zap - "this is helpful in cleaning up the crashed browsers"

BrowserControl()
define BrowserControl {
    comment("You will most almost certainly have to reset the “Crash Control” scrap text,
but this will have a lower CPU and Memory overhead then using WinAutomation.exe to close the crashed browser window and error message. 
Downside is if you are selling bots it will most lightly fail unless you find a way of getting X Y position of the browser crash error massage  ")
    set(#CrashControl, $plugin function("WindowsCommands.dll", "$scrape text", "Browser.exe", "#32770", 118, 18), "Global")
    if($contains(#CrashControl, "Browser.exe")) {
        then {
            plugin command("Advanced Shell.dll", "shell batch hidden", "TASKKILL /IM WerFault.exe /F")
            set(#CrashControl, $nothing, "Global")
        }
        else {
        }
    }
}

The above  code is good if you are using bots for personal use rather than selling it.

 

Useful third party software
http://w-shadow.com/blog/2009/03/04/restart-on-crash/
http://www.pcwintech.com/cleanmem
http://www.piriform.com/ccleaner/download

http://www.jockersoft.com/english/appmonitor_index.php

Use OS Addins memory cleaner node.
http://www.ubotstudio.com/forum/index.php?/topic/13444-sell-plugin-os-operating-system-addins/

The plugin comes with ready made  code we can plug into our bot for memeory cleaning

additionally it may be a good idea to let the user decide the frequency in seconds -for memory cleaning - via A Ui dropdown.

 

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Clearing cookies and cache

define Clear ALL Cookies {
    shell("C:\\windows\\system32\\rundll32.exe InetCpl.cpl,ClearMyTracksByProcess 4351")
    shell("cmd.exe /c rmdir /s /q \"%APPDATA%\\Macromedia\\Flash Player\\#SharedObjects\\\"")
    shell("C:\\Program Files\\CCleaner\\ccleaner.exe /AUTO")
}

-
-Right now the ONLY RELIABLE WAY to clear the cache is to start a new browser such as using the "in new browser" command.
-Needs clarification - read somewhere that 'clear cookies' command can lead to browser crashing - and that the workaround is a 'close page' node
to be used before it.? If someone could clarify.

-You can restart ubot as well to clear the cache of the main browser.
-using In new browser has numerous advantages -
                   All of these browsers have a new cache and cookies and will erase them when they close.
                   better memory management and more stable browser performance

 

Again from OS Addins we have these features clearing cookies and cache-
   - you can use the clear cache function in it and will clear all cache areas on the pc inlcuding the area ubot uses.
    -you can use the clear flash cookies node for clearing flash lso cookie/cache files
   - and the clear cookies node will clear all cookies system wide - I want to know what does this do exactly
and how is it relevant vis a vis UbotStudio's browser.
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

 

Misc browser related tips -

 

-Avoid using image recognition in a loop - this can cause browser.exe to crash very quickly.

 - I think UBot uses 1 "browser.exe" for UI, one for main UBot browser and additional process for each new UBot browser.

 

How To Get Ubot To Open Actual Browser Window-
-
shell("C:\\Program Files\\Internet Explorer\\iexplore.exe 'http://www.yahoo.com'")
shell("explorer.exe 'http://www.yahoo.com'")

shell("rundll32 url.dll,FileProtocolHandler http://www.ubotdev.com")

 

-
It is not recommended to use 'navigate' command inside on load("Tab Switch")
As the browser is not ready until bot is fully loaded.

(Please read UbotDev's update on this below.)
-
changing proxy inside In new browser - will set the proxy for that browser instance only.
so,use thread "in new browser" and then change the proxy as opposed to using "in shared browser"
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Popups/iframes related stuff -
----------------------------------
Example Popups in New Window - source
http://www.ubotstudio.com/forum/index.php?/topic/12812-example-popups-in-new-window-source/
-------------------------
links - in normal browser that would open in a new tab/browser, gets displayed like popup in UBot (which is actually an iframe)

To navigate to an iframe url -
scrape SRC parameter of that iframe window, and navigate to the URL scraped.
----
load html("<iframe src=\"http://www.facebook.com\"></iframe> ")
wait for browser event("Everything Loaded", "")
wait for element(<tagname="iframe">, "", "Appear")
wait(3)
navigate($scrape attribute(<tagname="iframe">, "src"), "Wait")
----
UBot displays popups inside iframes, and some sites like Facebook block their  display them inside iframes (that is why you see a blank popup).
---
to return to the page after finishing pop up tasks- navigate in a "new browser", so that once you are done with that, bot simply closes page and
gets you back to your main browser.
---
 

[UBotDev 276]

Wow, great research. Thanks for sharing it, I think it will help a lot of users to find that info in one place.

 

Btw, I never had problems with "clear cookie" command, where did you read that?

 

 

Here is what I can add:

 

Another useful shell command that will open URL in default browser:

shell("rundll32 url.dll,FileProtocolHandler http://www.ubotdev.com")

Using navigate command inside on load("Tab Switch") works just OK, it always loaded the page for me. I don't think the problem is that browser isn't ready, it's more that you have to ensure that navigate command doesn't fire if user is already running the bot (this would add unexpected navigate to execution, and bot would land on unexpected page). I solved that by adding another variable, which tells "on load" that bot is running, preventing the navigate to fire; however, by doing that another problem kicks in....if user hits stop I can't change that variable (it only gets changed on first and last line of the code), so navigate from "on load" won't fire anymore.

[LoWrIdErTJ - BotGuru 904]

very nice research and post bud..   heres some more info on the os addins plugin as well.

• you can use the clear cache function in it and will clear all cache areas on the pc inlcuding the area ubot uses
• you can use the clear flash cookies node for clearing flash lso cookie/cache files
• and the clear cookies node will clear all cookies system wide

[Sanjeev 46]

Thanks TJ ,Ubotdev.

 

Updated main post.

 

TJ? - would it be possible to have some sample code on how to use the  'Monitor Running Processes' node  for monitoring and closing crashed browser.exe instances?

 

thanks.

[LoWrIdErTJ - BotGuru 904]

Thanks TJ ,Ubotdev.

 

Updated main post.

 

TJ? - would it be possible to have some sample code on how to use the  'Monitor Running Processes' node  for monitoring and closing crashed browser.exe instances?

 

thanks.

 

 

its likely a lot easier to use this for that fix

http://www.ubotstudio.com/forum/index.php?/topic/13968-free-automatically-close-ubot-browserexe-crashes-fix/

[Sanjeev 46]

Thanks. Updated main post.

[Pete 121]

Some of you may find this helpful cleaning up the crashed browsers

BrowserControl()
define BrowserControl {
    comment("You will most almost certainly have to reset the “Crash Control” scrap text,
but this will have a lower CPU and Memory overhead then using WinAutomation.exe to close the crashed browser window and error message. 
Downside is if you are selling bots it will most lightly fail unless you find a way of getting X Y position of the browser crash error massage  ")
    set(#CrashControl, $plugin function("WindowsCommands.dll", "$scrape text", "Browser.exe", "#32770", 118, 18), "Global")
    if($contains(#CrashControl, "Browser.exe")) {
        then {
            plugin command("Advanced Shell.dll", "shell batch hidden", "TASKKILL /IM WerFault.exe /F")
            set(#CrashControl, $nothing, "Global")
        }
        else {
        }
    }
}

[Sanjeev 46]

Thanks for your Contribution. Main post updated.

[Sanjeev 46]

TJ?

 

is this worth adding in?

-------------------------------------------------------------------

a .vbs script for cleaning up dead browsers.

 

save the below code to a .vbs file
 

strComputer = "."
strProcessToKill = "Browser.exe"

SET objWMIService = GETOBJECT("winmgmts:" _
     & "{impersonationLevel=impersonate}!\\" _
     & strComputer & "\root\cimv2")

SET colProcess = objWMIService.ExecQuery _
     ("Select * from Win32_Process Where Name = '" & strProcessToKill & "'")

count = 0
FOR EACH objProcess in colProcess
     If objProcess.WorkingSetSize < 1048576 Then
     objProcess.Terminate()
     End If
NEXT

run the shell command to execute it when needed.

[Sanjeev 46]

http://www.ubotstudio.com/forum/index.php?/topic/12678-failed-loading-page/

Fellas? any ideas on workarounds for this issue?

Thanks.