2

I type this into the terminal:

$ scrapy startproject tutorial

I cannot get it to start a new scrapy project, and I keep installing all the things scrapy needs. I just can't get it to work. This is the error message it gives me:

File "/Users/carterdavis/anaconda/bin/scrapy", line 4, in <module>
execute()
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 122, in execute
cmds = _get_commands_dict(settings, inproject)
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 46, in _get_commands_dict
cmds = _get_commands_from_module('scrapy.commands', inproject)
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 29, in _get_commands_from_module
for cmd in _iter_command_classes(module):
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 20, in _iter_command_classes
for module in walk_modules(module_name):
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/utils/misc.py", line 68, in walk_modules
submod = import_module(fullpath)
File "/Users/carterdavis/anaconda/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/commands/bench.py", line 3, in <module>
from scrapy.tests.mockserver import MockServer
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/scrapy/tests/mockserver.py", line 6, in <module>
from twisted.internet import reactor, defer, ssl
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/twisted/internet/ssl.py", line 25, in <module>
from OpenSSL import SSL
File "/Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/__init__.py", line 36, in <module>
from OpenSSL import crypto
ImportError: dlopen(/Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/crypto.so, 10): Library not loaded: libssl.1.0.0.dylib
Referenced from: /Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/crypto.so
Reason: image not found

I have Python 2.7 and every package necessary to run scrapy.

Carter
  • 143
  • 6
  • http://stackoverflow.com/questions/11365619/psycopg2-installation-error-library-not-loaded-libssl-dylib possibly related – Robin Jan 23 '14 at 22:50

3 Answers3

4

Looks like you are using Anaconda - I had the same problem and this fixed it (tested on OS X 10.9 and 10.10).

First pip uninstall scrapy if you have it installed.

Use conda to install cryptography:

conda install cryptography

Set the DYLD_LIBRARY_PATH environment variable:

export DYLD_LIBRARY_PATH=$HOME/anaconda/lib

Then install scrapy again

pip install scrapy
Joe Hooper
  • 816
  • 5
  • 6
0

ImportError: dlopen(/Users/carterdavis/anaconda/lib/python2.7/site-packages/OpenSSL/crypto.so, 10):Library not loaded: libssl.1.0.0.dylib

According to the error message above, is there libssl package installed in your OS? If you are using Ubuntu(or Debian), try to execute there command:

apt-get install libssl1.0.0
Wayne
  • 26
  • 5
0

This is a problem of cryptography installation, try installing crytography again, which includes installation of OpenSSL also. And your project will starts working with this. Follow these steps.

For Debian and Ubuntu

$ sudo apt-get install build-essential libssl-dev libffi-dev python-dev

For Fedora and RHEL-derivatives

$ sudo yum install gcc libffi-devel python-devel openssl-devel

After installing this, you should now be able to build and install cryptography with the usual

$ pip install cryptography

After this, try creating your scrapy project.

Ravin Gupta
  • 783
  • 7
  • 13
  • Please don't [copy/paste](http://stackoverflow.com/questions/22073516/failed-to-install-python-cryptography-package-with-pip-and-setup-py/35498557#35498557) your answer to multiple questions. Customize the answer to the question and explain how this solves the problem. – Andy Feb 19 '16 at 06:30
  • 2 days back I was facing the same problem and followed these questions on stackoveflow. Now I founded the solution by myself and copy pasted solution to all those questions having wrong answers. – Ravin Gupta Feb 19 '16 at 06:37