107

I am trying to install lxml to install scrapy on my Mac (v 10.9.4)

╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  pip install lxml
Downloading/unpacking lxml
  Downloading lxml-3.4.0.tar.gz (3.5MB): 3.5MB downloaded
  Running setup.py (path:/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py) egg_info for package lxml
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.4.0.
    Building without Cython.
    Using build configuration of libxslt 1.1.28

    warning: no previously-included files found matching '*.py'
Installing collected packages: lxml
  Running setup.py install for lxml
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.4.0.
    Building without Cython.
    Using build configuration of libxslt 1.1.28
    building 'lxml.etree' extension
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace
    cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/etree.so
    building 'lxml.objectify' extension
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace
    cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/objectify.so
    error: could not create '/lib/python2.7/site-packages/lxml': Permission denied
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip-_xm20j-record/install-record.txt --single-version-externally-managed --compile:
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'

  warnings.warn(msg)

Building lxml version 3.4.0.

Building without Cython.

Using build configuration of libxslt 1.1.28

running install

running build

running build_py

creating build

creating build/lib.macosx-10.9-intel-2.7

creating build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/_elementpath.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/builder.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/cssselect.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/doctestcompare.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/ElementInclude.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/pyclasslookup.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/sax.py -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/usedoctest.py -> build/lib.macosx-10.9-intel-2.7/lxml

creating build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/includes

creating build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/_diffcommand.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/_html5builder.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/_setmixin.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/builder.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/clean.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/defs.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/diff.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/ElementSoup.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/formfill.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/html5parser.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/soupparser.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

copying src/lxml/html/usedoctest.py -> build/lib.macosx-10.9-intel-2.7/lxml/html

creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron

copying src/lxml/isoschematron/__init__.py -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron

copying src/lxml/lxml.etree.h -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/lxml.etree_api.h -> build/lib.macosx-10.9-intel-2.7/lxml

copying src/lxml/includes/c14n.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/config.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/dtdvalid.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/etreepublic.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/htmlparser.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/relaxng.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/schematron.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/tree.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/uri.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/xinclude.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/xmlerror.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/xmlparser.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/xmlschema.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/xpath.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/xslt.pxd -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/etree_defs.h -> build/lib.macosx-10.9-intel-2.7/lxml/includes

copying src/lxml/includes/lxml-version.h -> build/lib.macosx-10.9-intel-2.7/lxml/includes

creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources

creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/rng

copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/rng

creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl

copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl

copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl

creating build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.macosx-10.9-intel-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1

running build_ext

building 'lxml.etree' extension

creating build/temp.macosx-10.9-intel-2.7

creating build/temp.macosx-10.9-intel-2.7/src

creating build/temp.macosx-10.9-intel-2.7/src/lxml

cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace

cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/etree.so

building 'lxml.objectify' extension

cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -I/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/src/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -w -flat_namespace

cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.macosx-10.9-intel-2.7/lxml/objectify.so

running install_lib

creating /lib/python2.7/site-packages/lxml

error: could not create '/lib/python2.7/site-packages/lxml': Permission denied

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip-_xm20j-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /private/var/folders/8l/t7tcq67d34v7qq_4hp3s1dm80000gn/T/pip_build_ishaantaylor/lxml
Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log

If somebody could help me I would really appreciate it! I have a suspicion that the issue has to do with multiple versions of python but I'm not sure how to check or fix that.

Edit 1: When I try sudo pip install lxml:

╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  sudo pip install lxml
Traceback (most recent call last):
  File "/bin/pip", line 7, in <module>
    from pip import main
ImportError: No module named pip

Edit 2: When I try STATIC_DEPS=true sudo pip install lxml

╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  STATIC_DEPS=true sudo pip install lxml
Traceback (most recent call last):
  File "/bin/pip", line 7, in <module>
    from pip import main
ImportError: No module named pip

Also, I tried installing another package with pip and that didn't work either.. see below:

╭─ishaantaylor@Ishaans-MacBook-Pro.local /lib/python2.7/site-packages
╰─➤  pip install requests
Downloading/unpacking requests
  Downloading requests-2.4.1-py2.py3-none-any.whl (458kB): 458kB downloaded
Installing collected packages: requests
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 671, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 901, in move_wheel_files
    pycompile=self.pycompile,
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 215, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 205, in clobber
    os.makedirs(destdir)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/requests'
Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log

Edit 4: Tried to download virtualenv, people recommended this as an alternative to sudo. Result is below:

╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  pip install virtualenv                                                               127 ↵
Downloading/unpacking virtualenv
  Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded
Installing collected packages: virtualenv
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 671, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 901, in move_wheel_files
    pycompile=self.pycompile,
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 215, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 209, in clobber
    shutil.copy2(srcfile, destfile)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/virtualenv.py'

Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log

Edit 5:

╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  sudo pip install virtualenv                                                            1 ↵
Password:
Traceback (most recent call last):
  File "/bin/pip", line 7, in <module>
    from pip import main
ImportError: No module named pip
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  pip install virtualenv                                                                 1 ↵
Downloading/unpacking virtualenv
  Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded
Installing collected packages: virtualenv
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 671, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 901, in move_wheel_files
    pycompile=self.pycompile,
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 215, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg/pip/wheel.py", line 209, in clobber
    shutil.copy2(srcfile, destfile)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/lib/python2.7/site-packages/virtualenv.py'

Storing debug log for failure in /Users/ishaantaylor/Library/Logs/pip.log

Edit 6: In response to hd1:

╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  curl -kLso /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  sudo python get-pip.py
python: can't open file 'get-pip.py': [Errno 2] No such file or directory

Edit 7: In Response to hd1:

╭─ishaantaylor@Ishaans-MacBook-Pro.local /
╰─➤  curl -kLso /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py
╭─ishaantaylor@Ishaans-MacBook-Pro.local /
╰─➤  cd ~
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  sudo python /tmp/get-pip.py
Requirement already up-to-date: pip in /usr/local/lib/python2.7/site-packages/pip-1.5.6-py2.7.egg
Cleaning up...
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  sudo pip install virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-1.11.6-py2.py3-none-any.whl (1.6MB): 1.6MB downloaded
Installing collected packages: virtualenv
Successfully installed virtualenv
Cleaning up...
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  mkdir -p ~/.virtualenvs
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  virtualenv ~/.virtualenvs/lxmltest
Traceback (most recent call last):
  File "/bin/virtualenv", line 7, in <module>
    from virtualenv import main
ImportError: No module named virtualenv
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤  which virtualenv                                                       1 ↵
/bin/virtualenv
╭─ishaantaylor@Ishaans-MacBook-Pro.local ~
╰─➤ echo -e ${PATH//:/\\n}
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/opt/X11/bin
Simba
  • 23,537
  • 7
  • 64
  • 76
Ishaan Taylor
  • 1,817
  • 5
  • 17
  • 19
  • 2
    The answer is to do this: ```STATIC_DEPS=true pip install lxml```. No sudo! – nathancahill Sep 14 '14 at 02:59
  • 5
    When you edit your questions there's no need to include text like '**edit**' or '**update**' - there's an edit history feature that already covers this 99% of the time. – Flexo Sep 14 '14 at 07:20
  • The fact that you're trying to install into `/System/Library/…` means you're using a `pip` installed for Apple's Python, which usually ends up in `/usr/local/bin/pip`, but it could be anywhere else on your PATH; `which pip` will tell you. Meanwhile, the fact that `sudo pip` says that `pip` is not found means that whatever path `pip` is in is not part of the superuser's `PATH`, only your own `PATH`. You could try to fix that, or you could just `sudo /path/to/pip install lxml` instead. – abarnert Sep 23 '14 at 21:30
  • @abarnert I used `brew install python` to install pip + another python (total 2: default apple python and homebrew python) and when I run `which -a pip` I only get `/usr/local/bin/pip`. Is this really the default apple `pip`? I thought homebrew uses the `/usr/local/` directory – Ishaan Taylor Sep 24 '14 at 15:50
  • @IshaanTaylor: The problem is that both Apple Python _and_ Homebrew Python by default try to install runnable scripts in `/usr/local/bin`, and whichever `pip` (or `ipython`, etc.) you install second will overwrite the first. Try `less /usr/local/bin/pip`; if the shbang line is either `#!/System/…/python` or `#!/usr/bin/python`, it's the Apple Python; if it's either `#!/usr/local/Cellar/…/python` or `#!/usr/local/bin/python`, it's the Homebrew one. – abarnert Sep 24 '14 at 17:27
  • @IshaanTaylor: Based on the errors involving `/System/…`, either (a) you've got the Apple Python `pip`, (b) you've got it on your `PYTHONPATH` (try `echo $PYTHONPATH` to see), or (c) your Homebrew Python is hopelessly confused and trying to incorporate the Apple Python stdlib and site-packages (which I've never seen happen, but it's not inconceivable, so if you rule out the first two, this is next…). – abarnert Sep 24 '14 at 17:29
  • Anyway, installing `virtualenv` is almost certainly going to have the same problems as installing `lxml` in the first place, so I don't know why people are suggesting that as a solution. Yes, it's a great idea, and it means you'll only have to deal with this problem one time instead of over and over for every package, but you still have to solve it that one time… – abarnert Sep 24 '14 at 17:32
  • @abarnert when I try `echo $PYTHONPATH` there is a blank response. How can I check to see if your (a) is the case? – Ishaan Taylor Sep 24 '14 at 23:40
  • @IshaanTaylor: If it's blank, you've already checked it, and it's not (b). For (a), look at the previous comment: the first line in the `pip` program (which is just a short Python script) tells you which Python it was installed for. – abarnert Sep 24 '14 at 23:51

3 Answers3

186

It looks like you're having a permissions error, based on this message in your output: error: could not create '/lib/python2.7/site-packages/lxml': Permission denied.

One thing you can try is doing a user install of the package with pip install lxml --user. For more information on how that works, check out this StackOverflow answer. (Thanks to Ishaan Taylor for the suggestion)

You can also run pip install as a superuser with sudo pip install lxml but it is not generally a good idea because it can cause issues with your system-level packages.

Community
  • 1
  • 1
Kevin London
  • 4,628
  • 1
  • 21
  • 29
17

Set up a virtualenv:

% curl -kLso /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py 
% sudo python /tmp/get-pip.py

These commands install pip into the global site-packages directory.

% sudo pip install virtualenv

and ditto for virtualenv:

% mkdir -p ~/.virtualenvs

I like my virtualenvs under one tree in my home directory called .virtualenvs

% virtualenv ~/.virtualenvs/lxmltest

Creates a virtualenv.

% . ~/.virtualenvs/lxmltest/bin/activate

Removes the need to specify the full path to pip/python in this virtualenv.

% pip install lxml

Alternatively execute ~/.virtualenvs/lxmltest/bin/pip install lxml if you chose not to follow the previous step. Note, I'm not sure how far along you are, so some of these steps can be safely skipped. Of course, if you mess something up, you can always rm -Rf ~/.virtualenvs/lxmltest and start again from a new virtualenv.

lambinator
  • 10,616
  • 7
  • 55
  • 57
hd1
  • 33,938
  • 5
  • 80
  • 91
1

On a Mac, you need to use this command:

STATIC_DEPS=true sudo pip install lxml

Trane Francks
  • 134
  • 3
  • Tried this- please see result in edit 2 above. – Ishaan Taylor Sep 13 '14 at 00:08
  • this worked for me but I'm concerned the need for sudo means something isn't set up correctly – samaspin Apr 29 '20 at 08:13
  • @samaspin Unless you're installing packages into your user home structure, you will always need elevated privileges to install into system folders such as /usr/local/*. Being able to install into system folders without authenticating would itself indicate something isn't set up correctly. Requiring authentication for elevated privileges is an essential component of system security. – Trane Francks May 01 '20 at 01:04