0

I am trying to scrap a ecommerce site using scrapy and splash, as most of the content on the page is dynamic which is rendered by JS, I am using splash to fetch the data. I am able to fetch all the other data except the price data.

<div class="grid-row rendered">
    <span data-id="from-price" data-bind="visible: showFromLabel, text: fromLabelText, css: { 'price-from-discounted' : isDiscountedPrice }" class="price-from" style="display: none;">From</span>
    <span data-id="current-price" data-bind="text: priceText(), css: {'product-price-discounted' : isDiscountedPrice }" class="current-price">$31.58</span>
    <span data-id="rrp-price" data-bind="text: rrpPriceText, visible: showRrpPrice" class="product-prev-price" style="display: none;"></span>
    <span data-id="previous-price" data-bind="text: prevPriceText, visible: showPrevPrice" class="product-prev-price" style="display: none;"></span>
    <span data-id="german-vat-message" data-bind="text: germanVatMessage, visible: showGermanVatMessage" class="product-german-vat" style="display: none;"></span>
</div>

I want to fetch the 2nd span data, but the div.grid-row rendered is not rendered when I print(response.body).

How can I get that price data.

Here is the link to the page

Harkamal
  • 496
  • 3
  • 12

1 Answers1

0

Finally fixed the problem, I ran the web page inside scrapy shell with splash and then when I did view(response), it turned out that the price section was not loaded, then I googled around and found this answer, Basically, I just had to run the splash in private mode as per shown in the answer from the link.