Jump to content
UBot Underground

Scrape from flash website


Recommended Posts

I am trying to build a bot that will allow me to put in a url for an even on Stubhub.com and scrape the quantity of tickets remaining and the lowest price of the tickets for sale. Stubhub seems to use one of the following three types of listings for all events:

 

http://www.stubhub.com/kansas-city-chiefs-tickets/chiefs-vs-falcons-arrowhead-stadium-4076006/

http://www.stubhub.com/new-york-knicks-tickets/new-york-knicks-albany-times-union-center-10-20-2012-4078808/

http://www.stubhub.com/one-direction-tickets/one-direction-new-york-madison-square-garden-3-12-2012-4067933/

 

For the first type of url, I am able to just do a scrape and replace to get the quantity into a list and then I create a list from a list to scrape the list of href for the go buttons on there. Then I fill in the click target with list position 0, and that takes me to the listing data for the very first (lowest priced) tickets. From there I can just scrape the price into a list. So that format I've completely figured out, but would be open to solutions that didn't require me to click the go button and could just scrape the first price on the list.

 

So the biggest problem I'm having is that I can't figure out how to scrape either piece of data from the other two links. So first, is there anyway to get the ticket quantity out of that box. Second, how can I click on the first go button so that I can then scrape the price on the next screen?

 

Any help would be greatly appreciated, even if it's not necessarily ubot specific.

Link to post
Share on other sites

Unfortunately you can't scrape from a flash component, so there is no viable solution for that end of the process other than clicking the Go button.

 

John

Link to post
Share on other sites

Just for a better understanding of the way scraping works, why is flash so hard to scrape from? Is there absolutely no way to do it, or is it just that it's not possible to do using ubot? Also, why am I am able to scrape from the first link type, but not the other two. Is the first one not actually flash?

Link to post
Share on other sites

Thanks for the followup. I was able to find a developer on freelancer that built me a bot that could do that a while back, but he refused to provide support for the product, and it quit working. So it is possible. I just have no idea how to do it.

Link to post
Share on other sites

Flash ain't all that...

 

This takes care of the first one...http://magelabs.net/recommends/tickets%20available.png

And this takes care of the other...

 

http://magelabs.net/recommends/first%20go%20button.png

 

http://www.ubotstudio.com/forum/public/style_emoticons/default/rolleyes.gif

  • Like 1
Link to post
Share on other sites
  • 1 year later...

Just for a better understanding of the way scraping works, why is flash so hard to scrape from? Is there absolutely no way to do it, or is it just that it's not possible to do using ubot? Also, why am I am able to scrape from the first link type, but not the other two. Is the first one not actually flash?

Hi MK21,  I am looking for someone to scrape data from flash sites.  Were you able to solve your flash site issue? I saw on one of your posts you say you hired a developer to do this? 

 

Thnaks

Link to post
Share on other sites

I agree, I see no problem with scraping the sites mentioned. 

 

Here's an example:

navigate("http://www.stubhub.com/kansas-city-chiefs-tickets/", "Wait")
set(#event, $scrape attribute($element offset(<class="eventName">, 0), "innertext"), "Global")
set(#date, $scrape attribute($element offset(<class="date ticketDetails">, 0), "innertext"), "Global")
set(#ticketsleft, $scrape attribute($element offset(<itemprop="offerCount">, 0), "innertext"), "Global")
Link to post
Share on other sites

I don’t see any problem scrapping all data on those pages ....

Gentlemen, am I allowed to give you a flash website here that I want to scrape? Only issue is you have to create a demo account before seeing the interface that needs to be scraped.

Link to post
Share on other sites

 

I agree, I see no problem with scraping the sites mentioned. 

 

Here's an example:

navigate("http://www.stubhub.com/kansas-city-chiefs-tickets/", "Wait")
set(#event, $scrape attribute($element offset(<class="eventName">, 0), "innertext"), "Global")
set(#date, $scrape attribute($element offset(<class="date ticketDetails">, 0), "innertext"), "Global")
set(#ticketsleft, $scrape attribute($element offset(<itemprop="offerCount">, 0), "innertext"), "Global")

Or Can I PM you with the details?

Link to post
Share on other sites

Gentlemen, am I allowed to give you a flash website here that I want to scrape? Only issue is you have to create a demo account before seeing the interface that needs to be scraped.

 

You can post the site and we can take a look at it, but if the part you need to scrape is really flash, I don't think it will be possible to get without using some kind of screen capture and OCR.

Link to post
Share on other sites
  • 5 months later...

You can post the site and we can take a look at it, but if the part you need to scrape is really flash, I don't think it will be possible to get without using some kind of screen capture and OCR.

 

Been looking at this today. Have someone who wants to scrape data from a site with Adobe Stage Video. What I did was "save browser image" and then ran the image through this OCR site: http://www.onlineocr.net

 

Results are pretty accurate, some minor details would need cleaning up but nothing regex couldn't fix.

 

Would much prefer an all in one OCR plugin though if it existed!

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...