5

i am web scraping using scrapy and selenium webdrivers. But i find selenium webdriver to be very slow, but i find it easier to extract CSS properties of a webelement eg.

webElement.value_of_css_property('font-size')

how can this be achieved using scrapy only and no selenium webdriver.

alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
aman
  • 1,875
  • 4
  • 18
  • 27

1 Answers1

1

For this you have to have a rendered content in a real browser. Scrapy downloader is not a browser and has only the initial HTML page, there is no javascript engine built-in and it cannot download additional CSS or JS files.

With Scrapy you can only get the style attribute value of an element, but nothing more than that. selenium is the tool for the job.

Note that you can avoid having a real display as a dependency and automate phantomjs headless browser or other browsers in a virtual display.

Community
  • 1
  • 1
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195