Scraping Amazon Electronics Best Sellers

[stan beck 0]

Hello UBotters,

 

It's been a while since I've picked up UBot - I've a client that wants to scrape the urls for amazon best sellers and this is the code that I've come up with:

 

clear list(%urls)
navigate("http://www.amazon.com/gp/bestsellers/electronics/ref=sv_e_1", "Wait")
wait for browser event("Page Loaded", "")
add list to list(%urls, $scrape attribute(<class="zg_itemImmersion">, "href"), "Delete", "Global")

 

The page source for one of the item looks like this:

 

<div class="zg_itemImmersion"><div class="zg_rankDiv"><span class="zg_rankNumber">5.</span></div><div class="zg_itemWrapper" style="height:305px"><div class="zg_rankLine"><table><tr>

                      <td class=""></td>

                      <td class="zg_daysInList">63 days in the top 100</td>

                    </tr></table></div><div class="zg_image"><div class="zg_itemImageImmersion"><a  href="

 

 

 

 

 

 

http://www.amazon.com/Fire-HD-Display-Wi-Fi-GB/dp/B00KC6I06S/ref=zg_bs_electronics_5

"><img src="http://ecx.images-amazon.com/images/I/41gaAta1xDL._SL160_SL150_.jpg" alt="Fire HD 6, 6" HD Display, Wi-Fi, 8 GB..." title="Fire HD 6, 6" HD Display, Wi-Fi, 8 GB..."/></a></div></div><div class="zg_title"><a  href="

 

 

 

 

 

 

http://www.amazon.com/Fire-HD-Display-Wi-Fi-GB/dp/B00KC6I06S/ref=zg_bs_electronics_5

">Fire HD 6, 6" HD Display, Wi-Fi, 8 GB...</a></div><div class="zg_reviews">

<span class="crAvgStars" style="white-space:no-wrap;"><span class="asinReviewsSummary acr-popover" name="B00KC6I06S" ref="zg_bs_electronics_cm_cr_acr_pop_" >

             <a style="cursor:pointer;text-decoration:none" href="http://www.amazon.com/Fire-HD-Display-Wi-Fi-GB/product-reviews/B00KC6I06S/ref=zg_bs_electronics_cm_cr_acr_img?ie=UTF8&showViewpoints=1" name="reviewHistoPop_B00KC6I06S__star__" onmouseover="return acrPopoverHover(this,1);" onmouseout="return acrPopoverHover(this,0);" ><span class="swSprite s_star_4_0 " title="4.2 out of 5 stars" ><span>4.2 out of 5 stars</span></span> </a> <span class="histogramButton"

          style=";<a style="text-decoration:none" href="http://www.amazon.com/Fire-HD-Display-Wi-Fi-GB/product-reviews/B00KC6I06S/ref=zg_bs_electronics_cm_cr_acr_img?ie=UTF8&showViewpoints=1" name="reviewHistoPop_B00KC6I06S__button__" onmouseover="return acrPopoverHover(this,1);" onmouseout="return acrPopoverHover(this,0);" style="cursor:pointer" ><span class="swSprite s_chevron " ></span> </a></span></span>(<a href="http://www.amazon.com/Fire-HD-Display-Wi-Fi-GB/product-reviews/B00KC6I06S/ref=zg_bs_electronics_cm_cr_acr_txt?ie=UTF8&showViewpoints=1" >1,562</a>)</span></div><div class="zg_itemPriceBlock_compact"><div class="zg_price"> <strong class="price">$99.00</strong> <br />

 

 

 

 

</div></div></div><div class="zg_clear"></div></div>

 

Am I on the right path?

 

Thanks!

[itexspert 47]

Try This One you can add bigger waiting time if you want.

 

clear list(%urls)
navigate("http://www.amazon.com/gp/bestsellers/electronics/ref=sv_e_1", "Wait")
wait for browser event("Everything Loaded", "")
set(#scrape, $scrape attribute(<outerhtml=w"<a href=\"http://www.amazon.com*">, "href"), "Global")
add item to list(%urls, $list from text(#scrape, $new line), "Delete", "Global")

[stan beck 0]

Thanks for the reply itexpert - will give that a try and let you know how it works...

 

Stan