Jump to content
UBot Underground

:( I Suck At Scraping Data Tried Everything I Know


Recommended Posts

http://posting.albany.backpage.com/online/classifieds/PostAdPPI.html/alb/albany.backpage.com/?u=alb&serverName=albany.backpage.com&category=5416&section=4374

 

Guys when I make a script to navigate to each of the pages I have which is over 400 of them, I want to post to each biz op page for an actual job that I am offering for sales reps. Anyway I go to the page that you see above and now have to scrape each one and pick them each in a new navigation. Blah blah I can't scrape the href's and it's got me going crazy here is the code:

 

loop($list total(%urls)) {
    navigate($list item(%urls,#number),"Wait")
    click(<value="Post Ad">,"Left Click","No")
    wait for browser event("Everything Loaded","")
    wait(5)
    click(<data-name="services">,"Left Click","No")
    wait(5)
    click(<data-category=5416>,"Left Click","No")
    wait(5)
    add list to list(%chooselocation,$scrape attribute(<href=w"/online/classifieds/*;section=4374&category=5416">,"href"),"Delete","Global")
    increment(#number)

 

in the add list to list, there doesn't seem to be anything special about the url other than the section and category. I tried using wildcards but they dont work at all for me on this and when I try to use regular expressions they dont work because I dont know enough about them. What am i doing wrong?

 

I need the hrefs for each location at the above link but I am doing this for a specific number of states. I need tips on what to do. I have watched videos on youtube and havent found anything that has helped. :( 

 

 

Link to post
Share on other sites

Is this what your looking for, Done with wildcards. It  scrapes the cities in the url you gave.

 

add list to list(%urls$scrape attribute(<outerhtml=w"<a href=\"/online/classifieds/PostAdPPI.html/*/*.backpage.com/?u=*&serverName=*.backpage.com&superRegion=*&section=4374&category=5416\" data-superregion=\"*\" data-multiple=\"no\">*</a>">, "fullhref"), "Delete", "Global")

  • Like 1
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...