Jump to content
UBot Underground

Force Complete Page To Load; Wait For Ajax To Complete


Recommended Posts

I am trying to scrape data from a complex page which seems to have lots of dynamic AJAX running in the background.  Here is the page:

 

http://v3.torontomls.net/Live/Pages/Public/Link.aspx?Key=f7aa9fac8a5b45ed9f3baa480373d09a&App=TREB

 

As you will see from this page, there is a table at the top summarizing the 45 detail records which follow the table.  The problem I am having is that when I use the command,

wait for browser event("Everything Loaded", 30)

the page does not in fact load completely, but instead it seems to load the first half of the page, and then it jumps to the end.  When you look at the part of the page that is visible on the screen, you cannot tell that the complete page has not loaded.  Only if I manually scroll the full page from to bottom, does it force all of the AJAX elements on the page to load.  I also tried using this command:

run javascript("javascript:window.scrollTo(0, document.body.scrollHeight);")

but that doesn't seem to work either because it just jumps to the end and doesn't load the AJAX elements in the middle of the page. 

 

Any ideas on how to force the complete page to load?  Thanks.

 

 

Link to post
Share on other sites

not sure.

try.

navigate("http://v3.torontomls.net/Live/Pages/Public/Link.aspx?Key=f7aa9fac8a5b45ed9f3baa480373d09a&App=TREB","Wait")
wait for browser event("Everything Loaded","")
set(#height,$eval("var heigth =  document.body.scrollHeight;
heigth;"),"Global")
set(#loop,$divide(#height,500),"Global")
set(#loop,$add(#loop,10),"Global")
set(#i,500,"Global")
loop(#loop) {
    run javascript("javascript:window.scrollTo(0, {#i});")
    wait(0.05)
    set(#i,$add(#i,500),"Global")
}
wait(0.05)
run javascript("javascript:window.scrollTo(0, document.body.scrollHeight);")
Link to post
Share on other sites

Thanks for trying, but I'm getting "bad cast" error at: loop(#loop) { ...

 

Not sure why.  I'll keep playing with it.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...