Jump to content
UBot Underground

Is it possible to extract a page's html without having to go it physically


Recommended Posts

Hi fellas,

 

I'm trying to scrape a list of websites for a specific word, say "test drive" for example. Is it possible for ubot to extract a particular's site home page html code without having to go there and then scrape the <body> tag which takes a relatively long time.

 

thank in advance

Link to post
Share on other sites

Yep, as Zap said, you can you "read file" command if you only need to send GET requests.

 

The code would look like this:

set(#TEXT, $find regular expression($read file("http://www.ubotstudio.com/forum/index.php?/topic/16237-is-it-possible-to-extract-a-pages-html-without-having-to-go-it-physically/"), "test drive"), "Global")

Link to post
Share on other sites

You should also know that this command actually uses UBot's "browser.exe" browser to get the content.

 

On one side this is is a bad thing because of all problems related to UBot browser...

 

...but on the other side this allows you to easily get the content for which you have to log in, since you just do the log in to specific site inside UBot browser as you normally would (navigate->change attribute->click...) and then you can just use the read command to get the content for which authentication/authorization is required. I was actually using this as alternative to "http post" plugin to make "hybrid" bots with UBot native commands, which are running much faster.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...