4

I am using Python 3.6.3 and Pip 9.0.1 but still can't install scrapy? I am doing this on windows. When executing the following command

pip3 install scrapy

I am greeted with this error first..

----------------------------------------
Failed building wheel for Twisted
Running setup.py clean for Twisted
Failed to build Twisted
Installing collected packages: Twisted, scrapy
Running setup.py install for Twisted ... error

Then it continues, the second error stops it completly and seems a lot more fatal...

Command "c:\users\admin\appdata\local\programs\python\python36-32\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\admin\\AppData\\Local\\Temp\\pip-build-zxkenzjd\\Twisted\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\admin\AppData\Local\Temp\pip-tr72roue-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\admin\AppData\Local\Temp\pip-build-zxkenzjd\Twisted\

I have tried executing the following commands as suggested on this answer:

pip install -U setuptools
pip install -U wheel
t242987
  • 41
  • 1
  • 2
  • Did you use `pip3 install -U setuptools wheel` or simply `pip`? – Adam Dec 18 '17 at 22:18
  • I'm not sure, but trying with `pip3` now results in 'Requirement already up-to-date' so I'm pretty sure its all up to date. – t242987 Dec 18 '17 at 22:21
  • Yeah i'm not sure. I don't have windows. I'm not able to recreate using 3.6.3 on Ubuntu. – Adam Dec 18 '17 at 22:26
  • error shows it has problem with module `Twisted`. Maybe try to install `Twisted` with `pip`. You can also see for precompiled module on [Unofficial Windows Binaries for Python Extension Packages](https://www.lfd.uci.edu/~gohlke/pythonlibs/). Or use Anaconda distribution - it has `scrapy` on [its packages list](https://docs.anaconda.com/anaconda/packages/py3.6_win-64) so it could work correctly with Anaconda – furas Dec 18 '17 at 23:21

2 Answers2

6

I had the same problem too but I solved it as follow:

Open the Anaconda Prompt as administrator (For Windows10: open cortana/search Anaconda Prompt/choose Run as Administrator)

You should go to the path of Anaconda, for me was like:

C:\WINDOWS\system32>cd ..
C:\WINDOWS>cd..
C:\>cd  ProgramData
C:\ProgramData>cd Anaconda3
C:\ProgramData>Anaconda3>

Then you should run the following command

C:\ProgramData>Anaconda3>conda install -c anaconda twisted

At some point it asks

Proceed ([y]/n)?

type y. Now twisted is installed.

To install scrapy, you either install it in Anaconda Prompt (as administrator) by running the following command:

C:\ProgramData>Anaconda3>conda install -c conda-forge scrapy

(again y for Proceed ([y]/n)?)

or on jupyter notebook and run the command

!pip install scrapy
Monirrad
  • 465
  • 1
  • 7
  • 17
  • Just want to point out that Anaconda is not essential to the solution. Simply installing the compiled Twisted whl file as @Bishwas describes can solve this without needing Anaconda. – Steven Dec 16 '20 at 06:45
0

for my env: Win11 x64 + Python 3.11 x64

Solution:

crifan
  • 12,947
  • 1
  • 71
  • 56