0

How can we get all the referenced js and images from the website as here in addition to js and images referenced in the html.

We can see js and images in addition to mentioned in html in chrome developer tools Sources section. How can we acheive it programmatically?

Selva Saravana Er
  • 199
  • 1
  • 1
  • 6
  • If the images are not in the html source they are most likely populated by javascript and ajax request. Check out network tab in developer tools, there's most likely an ajax request being made that returns json of image urls or something like that. See related: http://stackoverflow.com/questions/8550114/can-scrapy-be-used-to-scrape-dynamic-content-from-websites-that-are-using-ajax – Granitosaurus Sep 28 '16 at 14:04
  • As a side note you can use scrapy-splash to get a HAR file that logs all the network into a json object. From there on you can download all images that are inbound from javascript requests. – Rafael Almeida Sep 28 '16 at 15:11

0 Answers0