Jump to content
UBot Underground

Twitter usernames scraping


Recommended Posts

i was tryin to scrape usernames of all the followers of a guy

everything was great and i was able to scrape the first 20 then i got stuck with the fact that in twitter you need to scroll down the page in order to see the rest of the followers list

 

so is there a way around that i can scrape all the followers usernames?

Link to post
Share on other sites
  • 2 weeks later...

Hey I like that bit of javascript...cheers John

 

Another way of doing this would be to retrieve the feed from the Twitter API:

 

here's how I do it:

 

clear list(%userids)
clear list(%usernames)
set(#doctext, $read file("https://api.twitter.com/1/followers/ids.xml?cursor=-1&screen_name=tweetmagnate"), "Global")
add list to list(%userids, $find regular expression(#doctext, "(?<=\\<id\\>).*(?=\\<)"), "Delete", "Global")
loop(50) {
   navigate("http://twitter.com/account/redirect_by_id?id={$next list item(%userids)}", "Wait")
   add item to list(%usernames, $url, "Delete", "Global")
   load html($text from list(%usernames, "<BR>"))
}

 

In the above example I retrieve 5000 user ids of tweetmagnate's followers

I then get the username by navigating to twitter. in the above example I'd get 50 usernames

 

I've used similar routines for following and unfollowing.

 

Hope this helps.

 

Cheers

 

Rob

 

 

i was tryin to scrape usernames of all the followers of a guy

everything was great and i was able to scrape the first 20 then i got stuck with the fact that in twitter you need to scroll down the page in order to see the rest of the followers list

 

so is there a way around that i can scrape all the followers usernames?

  • Like 1
Link to post
Share on other sites
  • 1 year later...

This is great magzmedia, but unfortunately it no longer works now that Twitter has updated their API. Do you have a solution for the new API? Thanks!

 

Hey I like that bit of javascript...cheers John

Another way of doing this would be to retrieve the feed from the Twitter API:

Link to post
Share on other sites
  • 1 year later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...