mk21 1 Posted June 7, 2012 Report Share Posted June 7, 2012 I am trying to build a bot that will allow me to put in a url for an even on Stubhub.com and scrape the quantity of tickets remaining and the lowest price of the tickets for sale. Stubhub seems to use one of the following three types of listings for all events: http://www.stubhub.com/kansas-city-chiefs-tickets/chiefs-vs-falcons-arrowhead-stadium-4076006/http://www.stubhub.com/new-york-knicks-tickets/new-york-knicks-albany-times-union-center-10-20-2012-4078808/http://www.stubhub.com/one-direction-tickets/one-direction-new-york-madison-square-garden-3-12-2012-4067933/ For the first type of url, I am able to just do a scrape and replace to get the quantity into a list and then I create a list from a list to scrape the list of href for the go buttons on there. Then I fill in the click target with list position 0, and that takes me to the listing data for the very first (lowest priced) tickets. From there I can just scrape the price into a list. So that format I've completely figured out, but would be open to solutions that didn't require me to click the go button and could just scrape the first price on the list. So the biggest problem I'm having is that I can't figure out how to scrape either piece of data from the other two links. So first, is there anyway to get the ticket quantity out of that box. Second, how can I click on the first go button so that I can then scrape the price on the next screen? Any help would be greatly appreciated, even if it's not necessarily ubot specific. Quote Link to post Share on other sites
JohnB 255 Posted June 12, 2012 Report Share Posted June 12, 2012 Unfortunately you can't scrape from a flash component, so there is no viable solution for that end of the process other than clicking the Go button. John Quote Link to post Share on other sites
mk21 1 Posted June 16, 2012 Author Report Share Posted June 16, 2012 Just for a better understanding of the way scraping works, why is flash so hard to scrape from? Is there absolutely no way to do it, or is it just that it's not possible to do using ubot? Also, why am I am able to scrape from the first link type, but not the other two. Is the first one not actually flash? Quote Link to post Share on other sites
mk21 1 Posted June 20, 2012 Author Report Share Posted June 20, 2012 Thanks for the followup. I was able to find a developer on freelancer that built me a bot that could do that a while back, but he refused to provide support for the product, and it quit working. So it is possible. I just have no idea how to do it. Quote Link to post Share on other sites
Legend 181 Posted June 20, 2012 Report Share Posted June 20, 2012 Flash ain't all that... This takes care of the first one...http://magelabs.net/recommends/tickets%20available.pngAnd this takes care of the other... http://magelabs.net/recommends/first%20go%20button.png http://www.ubotstudio.com/forum/public/style_emoticons/default/rolleyes.gif 1 Quote Link to post Share on other sites
shadsy 0 Posted October 23, 2013 Report Share Posted October 23, 2013 Just for a better understanding of the way scraping works, why is flash so hard to scrape from? Is there absolutely no way to do it, or is it just that it's not possible to do using ubot? Also, why am I am able to scrape from the first link type, but not the other two. Is the first one not actually flash?Hi MK21, I am looking for someone to scrape data from flash sites. Were you able to solve your flash site issue? I saw on one of your posts you say you hired a developer to do this? Thnaks Quote Link to post Share on other sites
brusacco 20 Posted October 23, 2013 Report Share Posted October 23, 2013 I don’t see any problem scrapping all data on those pages .... Quote Link to post Share on other sites
Steve 30 Posted October 23, 2013 Report Share Posted October 23, 2013 I agree, I see no problem with scraping the sites mentioned. Here's an example: navigate("http://www.stubhub.com/kansas-city-chiefs-tickets/", "Wait") set(#event, $scrape attribute($element offset(<class="eventName">, 0), "innertext"), "Global") set(#date, $scrape attribute($element offset(<class="date ticketDetails">, 0), "innertext"), "Global") set(#ticketsleft, $scrape attribute($element offset(<itemprop="offerCount">, 0), "innertext"), "Global") Quote Link to post Share on other sites
shadsy 0 Posted October 24, 2013 Report Share Posted October 24, 2013 I don’t see any problem scrapping all data on those pages ....Gentlemen, am I allowed to give you a flash website here that I want to scrape? Only issue is you have to create a demo account before seeing the interface that needs to be scraped. Quote Link to post Share on other sites
shadsy 0 Posted October 24, 2013 Report Share Posted October 24, 2013 I agree, I see no problem with scraping the sites mentioned. Here's an example: navigate("http://www.stubhub.com/kansas-city-chiefs-tickets/", "Wait") set(#event, $scrape attribute($element offset(<class="eventName">, 0), "innertext"), "Global") set(#date, $scrape attribute($element offset(<class="date ticketDetails">, 0), "innertext"), "Global") set(#ticketsleft, $scrape attribute($element offset(<itemprop="offerCount">, 0), "innertext"), "Global") Or Can I PM you with the details? Quote Link to post Share on other sites
Steve 30 Posted October 24, 2013 Report Share Posted October 24, 2013 Gentlemen, am I allowed to give you a flash website here that I want to scrape? Only issue is you have to create a demo account before seeing the interface that needs to be scraped. You can post the site and we can take a look at it, but if the part you need to scrape is really flash, I don't think it will be possible to get without using some kind of screen capture and OCR. Quote Link to post Share on other sites
Kev 69 Posted April 15, 2014 Report Share Posted April 15, 2014 You can post the site and we can take a look at it, but if the part you need to scrape is really flash, I don't think it will be possible to get without using some kind of screen capture and OCR. Been looking at this today. Have someone who wants to scrape data from a site with Adobe Stage Video. What I did was "save browser image" and then ran the image through this OCR site: http://www.onlineocr.net Results are pretty accurate, some minor details would need cleaning up but nothing regex couldn't fix. Would much prefer an all in one OCR plugin though if it existed! Quote Link to post Share on other sites
Steve 30 Posted April 15, 2014 Report Share Posted April 15, 2014 +1 on an OCR plugin Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.