0

Im trying to scrape a site that requires logging in. so i login.

The pages directly after the login seem to have alot of javascript.

How can i test out the responses of splash on scrapy shell?

Once i login, how can i run splash on the next url via splash command line and have it process the javascript and give me a response i can parse?

I dont understand what i actually need to do to run the splash service and how it functions with scrapy... Please point me in the right direction.

joe
  • 73
  • 2
  • 8
  • Duplicate of: https://stackoverflow.com/questions/35352423 ? – Granitosaurus Dec 19 '18 at 14:08
  • 1
    Possible duplicate of [Scrapy Shell and Scrapy Splash](https://stackoverflow.com/questions/35352423/scrapy-shell-and-scrapy-splash) – Granitosaurus Dec 19 '18 at 15:25
  • Regarding your question part that is not duplicate on how to run splash check out official installation documentation: https://splash.readthedocs.io/en/stable/install.html In short Splash is a http service that you install, run and then connect from your apps to render some web pages – Granitosaurus Dec 19 '18 at 15:29

0 Answers0