Jump to content
UBot Underground

DjProg

Fellow UBotter
  • Content Count

    86
  • Joined

  • Last visited

Community Reputation

3 Neutral

About DjProg

  • Rank
    Advanced Member

Profile Information

  • Gender
    Not Telling

System Specs

  • OS
    Windows 8
  • Total Memory
    < 1Gb
  • Framework
    v3.5
  • License
    Standard Edition

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. I would bet just doing a multithread "browsing 200 sites" bot will crash with v39... and from all the reports on the forum it's not a usable version. Yet I'll try to see if I can find what version of my bot had this issue, i.e. at what point I switched to v21.
  2. Well now I'm dumping my table in CSV for each loop, this way if Ubot goes crazy I'll still know at what loop it crashed and still have the results before the crash.
  3. Well by looking at the debugger it's not even that much: 28000 rows and 8 columns... What's wrong again with Ubot ? => IS THERE A WAY TO "SAVE MY DEBUGGER DATA" ?? Because clearly the data is still there...
  4. Hello guys, Another problem for no apparent reason: System.OutOfMemoryException was thrown... when doing a Save to file of a table of only MAX 50K lines !! Clearly this shouldn't happen, and when I check the system monitor, I can confirm that there is a TON of memory available. That's the system monitor when running the node... http://screencast.com/t/UEXhuP58 Needless to say that there is plenty of room to save a d*mn text file. Any idea ? I lost 3 hours+ of scrape due to this bug. I'm trying not to look to upset but I can tell you I AM !! Cheers,
  5. Update: thanks again Dan, the bloody v39 Ubot browser IS 100% the culprit... changed to v21 and it works flawlessly... Only downside of this is... can't change the Http Browser Headers with this old version.
  6. Thanks Dan, I'll try this and report back. I don't think my multithread is the culprit as even if I run my old (unthreaded) bot, it gets the white browser screen of death after visiting maybe 40 URLs. By the way I'm using 5.9.18, I hope it's not like a couple years ago, a.k.a. current version is buggy like hell and I would need to use V4 (?)
  7. ...until there is simply not a single working browser in any thread ! I'm wondering what the issue is here. => Could it be that the "navigate" function with "wait" enabled would crash if somehow the page doesn't load ? (of course I've visited the suspicious pages and they seem to work fine).. I'm expecting it to timeout instead of resulting in a "forever loading browser of dead) but maybe that's the cause (?). As it's a scrapper I don't see much which could go wrong, especially crash the browsers. Any idea ? I'm pretty sure it's a very common problem. Cheers,
  8. Thanks Dan. I've had to put 1 second instead of 0.5 but it's working now (well only the multithread issue I had... now I have the browser hanging / not responding after visiting a few dozens sites... "the white browser forever loading wheel of death" I've opened a ticket for this as it doesn't seem normal at all) Cheers,
  9. Tested again with 3 threads but a bigger test URLs list : http://google.comhttp://amazon.comhttp://yahoo.comhttp://bing.comhttp://ebay.comhttp://www.booking.com/https://www.airbnb.comhttp://www.alexa.com/https://login.live.com/https://uk.linkedin.com/https://www.mozilla.org/http://www.apple.com/http://www.linux.org/ And it's the complete havoc: 2016-04-09 14:16:46 [LOG] End crawling>>> http://yahoo.com2016-04-09 14:16:47 [LOG] End crawling>>> http://yahoo.com2016-04-09 14:16:49 [LOG] End crawling>>> http://yahoo.com2016-04-09 14:16:55 [LOG] End crawling>>>
  10. Hello guys, I have a somewhat funky behavior when running my multithread script here : reset browser clear cookies ui drop down("Max threads","1,2,3,4,5,6,7,8,9,10",#max_threads) ui block text("URLs to crawl",#ui_URLs) clear list(%urls) add list to list(%urls,$list from text(#ui_URLs,$new line),"Delete","Global") set(#url_crawling_position,"-1","Global") set(#used_threads,0,"Global") loop($list total(%urls)) { loop while($comparison(#used_threads,">= Greater than or equal to",#max_threads)) { wait(1) } loop_process() } define loop_process { increment(#used_threads
  11. Thanks ! I forgot to say but I'm adding the scraped attributes to a List. So after adding to list I would need to loop thought my list to "replace" the dirty innerHTML into the "cleaned", regexed text ? Or it there a more elegant solution ? CHeers,
  12. Hello guys, What is the best way to "clean" an innerHTML scraped attribute ? Basically I'm scraping an innerHTML containing an empty div "inline styled", for which I need to find the inline styled background-image URL... <div class="blablah" style="height:120px;background-image:url(http://somewhere.com/image.jpeg)"></div> I scraped the innerHTML of the parent div of blahblah because else I didn't get what I needed, but now I need to clean up a bit. Any tip is welcome ! Thanks a lot, Cheers,
  13. Hello, Unfortunately this simply doesn't work at all. Somehow the click dialog doesn't work with this modal window, it simply doesn't seem to do anything: the modal window appear and stay there... Any other idea? Cheers,
  14. Hello guys, Since 2 days I keep getting a tons of licensing server issues like that: http://screencast.com/t/qQmAJ37jCza Obviously everytime the licensing server is down, the support is down too. VERY annoying... What's the ETA to solve this ? I can't believe the system isn't redundant. Cheers,
×
×
  • Create New...