Scrapy Installation - Issue with lxml install

Question

I am new to Python and I am trying to install scrapy, I have python 2.7.12 and pip 8.1.2 on windows 10. when I give the command 'pip install scrapy' it tries to install lxml and gives the below error. I downloaded the libxml2 binary, extracted to a folder and added the bin folder in path variable.But still the same issue. Please guide me with this issue, I am stuck with this. The error message is mentioned below.

cl : Command line warning D9025 : overriding '/W3' with '/w'
lxml.etree.c
src\lxml\includes\etree_defs.h(14) : fatal error C1083: Cannot open include file: 'libxml/xmlversion.h': No such file or directory
Compile failed: command 'C:\\Users\\myuserid\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\cl.exe' failed with exit status 2
creating users
creating users\myuser~1
creating users\myuser~1\appdata
creating users\myuser~1\appdata\local
creating users\myuser~1\appdata\local\temp
C:\Users\myuserid\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I/usr/include/libxml2 /Tcc:\users\myuser~1\appdata\local\temp\xmlXPathInita7i8_a.c /Fousers\myuser~1\appdata\local\temp\xmlXPathInita7i8_a.obj
xmlXPathInita7i8_a.c
c:\users\myuser~1\appdata\local\temp\xmlXPathInita7i8_a.c(1) : fatal error C1083: Cannot open include file: 'libxml/xpath.h': No such file or directory
*********************************************************************************
Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
*********************************************************************************
error: command 'C:\\Users\\myuserid\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\cl.exe' failed with exit status 2

----------------------------------------

Command "c:\softwares\python27\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\myuser~1\\appdata\\local\\temp\\pip-build-m4bvsr\\lxml\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record c:\users\myuser~1\appdata\local\temp\pip-dnttln-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\myuser~1\appdata\local\temp\pip-build-m4bvsr\lxml\

Answer 1

you can simply use one of these whl files, based on your system.

After download it, use these command in where you download that file:

pip install ***.whl

Answer 2

Scrapy's docummentation recommends using Anaconda to install the package. Lxml package is really tricky to install on windows and anaconda is a really straigh-forward shortcut.

There's also unnoficial binaries of lxml mentioned in lxml's docummentation that you can use to install already compiled lxml package. There are more instructions as well, how to install lxml on windows os there too.

Answer 3

What version of pip do you have? Just upgrading pip helps in my case

pip install --upgrade pip