Scrape dynamic data using scrapy

Question

I would like to scrape option chain of stock from nasdaq website using scrapy (along with other data)

Nasdaq recently updated their website. Here is the url I am talking about.

The data is not loaded with plain spider and in scrapy shell. From the scrapy docs, I need to use scrapy-splash or scrapy-selenium.

I would like to know which one is the right tool for the job before investing my time. Or is there something else that is recommended?

Thanks!

https://stackoverflow.com/questions/49891688/does-using-scrapy-splash-significantly-affect-scraping-speed — Adrien Blanquer, Aug 20 '19 at 12:57

score 1 · Accepted Answer · answered Aug 20 '19 at 13:05

1

For that task you don't need selenium or splash. Instead you can inspect the network tab in chrome developer tools.

Cheers

answered Aug 20 '19 at 13:05

bace bogo

1 Answers1