Jump to content
UBot Underground

Big Slow Down On Scrape


Recommended Posts

I have something odd going on and can not figure out why it is happening. 

 

When I navigate through a bunch of profiles with the scrape define below included in the loop it slows down from taking a few seconds to 3-5 minutes at somewhere between 100-150 profiles. 

 

 

If I run the loop without the scrape define included it works just fine without slowing down and I have taken it to over 3k profiles viewed. 

 

The scrape is activated with a checkbox and an if

 

Any thoughts on a fix would be great. 

 

Thanks

 

 

         if($comparison(#Scrape_Profiles"=""true")) {
            then {
                Scrape()
            }
            else {
            }
        }

 

 

define Scrape {
    wait(.25)
    set(#scrape_name$scrape attribute(<class="full-name">"innertext"), "Global")
    wait(.25)
    set(#scrape_location$scrape attribute(<name="location">"innertext"), "Global")
    wait(.25)
    set(#scrape_company$scrape attribute($element offset(<href=w"/vsearch/p?company=*">, 0), "innertext"), "Global")
    click(<innerhtml="Contact Info">"Left Click""No")
    wait(1)
    set(#scrape_email$scrape attribute(<href=w"mailto:*">"innertext"), "Global")
    wait(.25)
    set(#scrape_phone$scrape attribute(<id=w"relationship-phone-number-*">"innertext"), "Global")
    wait(.25)
    if($exists(<class="view-public-profile">)) {
        then {
            set(#scrape_profile$scrape attribute(<class="view-public-profile">"innertext"), "Global")
        }
        else {
            set(#scrape_profile$scrape attribute(<href=w"http://https://*.com/*">"innertext"), "Global")
        }
    }
    wait(.25)
    set(#scrape_address$scrape attribute(<id=w"relationship-address-*">"innertext"), "Global")
    wait(.25)
    append to file("{$special folder("Desktop")}\\saved\\scrape.csv""{$new line}{#scrape_name};{#scrape_profile};{#scrape_location};{#scrape_company};\"{#scrape_email}\";\"{#scrape_phone}\";{#scrape_address}""End")
}

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...