0

I've been looking around and I can't seem to find an answer on how to run scrapyd in Python 3 and above. When I run it it keeps defaulting to python 2.7, though I recall reading in the docs or elsewhere that scrapyd supports python3: https://scrapyd.readthedocs.io/en/stable/news.html#id1

Running on an AWS Ubuntu 18.04 server. What am I doing wrong? How can I change to Python 3.4 and above?

I'm having problems because I'm using abstract inheritance in my spiders which python 2.7 doesn't support.

Edit: I'm able to run Scrapyd in Python 3 using a virtual environment, but how would you do it without one?

Thorvald
  • 546
  • 6
  • 18

1 Answers1

0

Simplest solution is use a virtual enviroment.

Since you are using Ubuntu and you can make it work using a venv I assume you installed scrapy using pip instead of pip3. Since Py2.7 is the stardard reference to python in Ubuntu, when running pip it will execute pip for py2 instead of py3.

Another way would be uninstalling pip uninstall scrapy and installing it again using pip3 pip3 install scrapy. This should fix the reference to scrapy. If by any chance you also use scrapy in py2.7 and therefore can't uninstall, then you have another reason to use venv.

renatodvc
  • 2,526
  • 2
  • 6
  • 17