[Help] Scraping Data Via Http

[seana88 2]

I just recently purchased Pro edition and really would like to learn how to scrape via HTTP with Aymen's Post Plugin. I have searched all the threads and read through all his documentation, but I would really like to learn how to scrape and can't seem to figure out how to (Regex confuses me). Could anyone work with me on how to scrape via HTTP?

 

For example, I'd like to be able to go to http://tinychat.com/ and scrape everything between ">#thetextiwanttoscrape</a></strong> via HTTP.

The text after the # is just the name of the tinychat rooms listed on the directory.

 

At least an example could help get in the right direction, as I want to move away from webbrowsers altogether. 

 

 

Thank you for your time,

Sean

[HelloInsomnia 1103]

I think this is what you want, it uses the xpath parser. Regex isn't really necessary here but it can also be used:

set(#tinychat, $plugin function("HTTP post.dll", "$http get", "http://tinychat.com/", "", "", "", ""), "Global")
clear list(%rooms)
add list to list(%rooms, $plugin function("HTTP post.dll", "$xpath parser", #tinychat, "//div[@class=\'room\']/strong/a", "InnerText", "HTML"), "Delete", "Global")

Dan has a HTTP Post course http://www.ubotstudio.com/forum/index.php?/topic/16544-sell-http-plugin-tutorial-video-series/

 

That may be able to help you learn more. I have a regex course and regex builder as well on my website if you're interested in regex. 

 

The community here is pretty good as well so if you ever have questions we are here to help.

[seana88 2]

Wow HelloImsomnia I just tested this out and it works! Thank you very much for showing me how to do a simple HTTP scrape. You and TJ have been great help to me in regards to the HTTP Plug-in. 

I think when I get paid next I will definitely pay for Dan's tutorial videos as well.

 

 Thanks again HelloImsonia, it means a lot to me.