APTS 3 Posted August 3, 2015 Report Share Posted August 3, 2015 I am trying to scrape data from a complex page which seems to have lots of dynamic AJAX running in the background. Here is the page: http://v3.torontomls.net/Live/Pages/Public/Link.aspx?Key=f7aa9fac8a5b45ed9f3baa480373d09a&App=TREB As you will see from this page, there is a table at the top summarizing the 45 detail records which follow the table. The problem I am having is that when I use the command, wait for browser event("Everything Loaded", 30)the page does not in fact load completely, but instead it seems to load the first half of the page, and then it jumps to the end. When you look at the part of the page that is visible on the screen, you cannot tell that the complete page has not loaded. Only if I manually scroll the full page from to bottom, does it force all of the AJAX elements on the page to load. I also tried using this command: run javascript("javascript:window.scrollTo(0, document.body.scrollHeight);")but that doesn't seem to work either because it just jumps to the end and doesn't load the AJAX elements in the middle of the page. Any ideas on how to force the complete page to load? Thanks. Quote Link to post Share on other sites
pash 504 Posted August 3, 2015 Report Share Posted August 3, 2015 not sure.try. navigate("http://v3.torontomls.net/Live/Pages/Public/Link.aspx?Key=f7aa9fac8a5b45ed9f3baa480373d09a&App=TREB","Wait") wait for browser event("Everything Loaded","") set(#height,$eval("var heigth = document.body.scrollHeight; heigth;"),"Global") set(#loop,$divide(#height,500),"Global") set(#loop,$add(#loop,10),"Global") set(#i,500,"Global") loop(#loop) { run javascript("javascript:window.scrollTo(0, {#i});") wait(0.05) set(#i,$add(#i,500),"Global") } wait(0.05) run javascript("javascript:window.scrollTo(0, document.body.scrollHeight);") Quote Link to post Share on other sites
APTS 3 Posted August 3, 2015 Author Report Share Posted August 3, 2015 Thanks for trying, but I'm getting "bad cast" error at: loop(#loop) { ... Not sure why. I'll keep playing with it. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.