4

I am trying to install the Scrapy module for Python. I am using the command:

pip install Scrapy

But I get the following error:

Command "/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools, tokenize;__file__='/private/var/folders/b4/nrmj5jf105lcv1_kf1yk9gh40000gn/T/pip-build-aNQoXA/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/b4/nrmj5jf105lcv1_kf1yk9gh40000gn/T/pip-B_akAd-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/b4/nrmj5jf105lcv1_kf1yk9gh40000gn/T/pip-build-aNQoXA/lxml

I've done some research on this, and tried several different solutions here and here but to no avail.

Community
  • 1
  • 1
123
  • 8,733
  • 14
  • 57
  • 99

2 Answers2

6

you can install it via pip wheel.
Download from here wheel package. Scrapy

Then install with

pip install wheel       #  to install support for wheel files.
pip install Scrapy‑1.0.3‑py2‑none‑any.whl        # to install the wheel.

EDIT: If on ubuntu or another debian based os:

try:

sudo apt-get install python-dev

sudo apt-get install libxml2-dev libxslt1-dev

and then again try to install scrapy.

On Mac Osx :

Install brew package manager from :Brew

Then try to use brew install package_name

Manish Gupta
  • 4,438
  • 18
  • 57
  • 104
  • I installed wheel, and upgraded pip, but now I'm getting the error `Failed building wheel for lxml` and also still get the same error I was originally. – 123 Sep 02 '15 at 19:40
  • Are you on linux or windows? Also tell which version. – Manish Gupta Sep 03 '15 at 04:41
  • The command `brew install scrapy` returns: `Error: No available formula for scrapy` – 123 Sep 03 '15 at 21:44
  • you have to install the dev packages, `python-dev libxml2-dev libxslt1-dev` from brew. You can install scrapy from pip or wheel – Manish Gupta Sep 04 '15 at 04:38
  • Error: No available formula for python-dev ==> Searching formulae... ==> Searching taps... – 123 Sep 06 '15 at 01:16
  • Im on Debian 8.5 and Im using python 3, all I had to do was install what you said but instead of `python-dev` i used `python3-dev` – Laraconda Jun 28 '16 at 19:33
2

Apparently I needed to install the Xcode command line tools. (Even if you have Xcode installed.) I have the latest beta of Xcode, but I still needed to install the command line tools.

xcode-select --install

And then running pip install scrapy worked fine.

123
  • 8,733
  • 14
  • 57
  • 99