APTS 3 Posted August 25, 2015 Report Share Posted August 25, 2015 (edited) I am trying to use Aymen's HTTP Post plugin but I cannot seem to grab the document text. I am getting an error: The remote server returned an error: (500) Internal Server Error.The first part of my code navigates to the desired site, then I enter login information and search parameters. The next step is to loop through the search results, which I am trying to do using the $http get command of the HTTP post plugin. Here is my code: clear all datanavigate("http://targetsite.com/?page=login","Wait")type text(<username field>,"username","Standard")type text(<password field>,"password","Standard")click(<login button>,"Left Click","No")wait(1)click(<innertext="Start Search">,"Left Click","No")wait(1)change dropdown(<name="startyr">,2015)wait(1)click(<value="search">,"Left Click","No")wait(1)set(#position,0,"Global")loop(3) { set(#x,$plugin function("HTTP post.dll", "$http get", targetsite.com/?page=details&skip={#position}", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36", "", "", ""),"Global") Scrape_Data() plugin command("HTTP post.dll", "http file download", targetsite.com/?page=details&skip={#position}", "C:\\myfolder\\{#myfilename}.html", "") increment(#position)} Can someone explain why I am getting the Internal Server Error? Thanks. One final related question: I recently updated to version 5.8.5 of the software and I am glad that I did. Pages now load incredibly fast, and overall this version seems more stable. One of the new features is the ability to do "headless browsing". What is the difference between headless browsing, and the `$http get` command from Ayment's HTTP Post plugin? Are they both essentially the same thing? Thanks again. Edited August 25, 2015 by APTS Quote Link to post Share on other sites
addy196 9 Posted August 25, 2015 Report Share Posted August 25, 2015 (edited) loop(1) { set(#x,$plugin function("HTTP post.dll", "$http get", "http://targetsite.com/?page=details&skip={#position}", $plugin function("HTTP post.dll", "$http useragent string", "Firefox 27.0 Win7 64-bit"), "", "", ""),"Global") plugin command("HTTP post.dll", "http file download", "http://targetsite.com/?page=details&skip={#position}", "C:\\\\myfolder\\\\{#myfilename}.html", "") increment(#position) } Try This Edited August 25, 2015 by addy196 Quote Link to post Share on other sites
APTS 3 Posted August 25, 2015 Author Report Share Posted August 25, 2015 Thanks for trying addy196, but I still get the Internal Server Error using your code. Quote Link to post Share on other sites
Aymen 385 Posted August 25, 2015 Report Share Posted August 25, 2015 The browser and the http plugin doesn't use the same cookies container , so you need to login with the http plugin in order to access a page behind a login! Regards Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.