12

I have virtualenv with --no-site-packages option. I'm using scrapy in it. Scrapy uses libxml2 by import libxml2. How to install libxml2 in virtualenv using pip or easy_install?

Tomasz Wysocki
  • 11,170
  • 6
  • 47
  • 62

6 Answers6

20

pip install ftp://xmlsoft.org/libxml2/python/libxml2-python-2.6.9.tar.gz

turingmachine
  • 499
  • 4
  • 9
  • 1
    for latest version ftp://xmlsoft.org/libxml2/python/libxml2-python-2.6.21.tar.gz – MechanisM Jun 04 '12 at 12:05
  • 1
    I got this error:failed to find headers for libxml2: update includes_dir – Danfi Jul 20 '12 at 01:40
  • 2
    @Danfi: You still need to install libxml2 and its headers: `sudo apt-get install libxml2 libxml2-dev` – turingmachine Jul 30 '12 at 11:54
  • Anyone know how I can place this link in my requirements.txt? I'm getting an error when doing -e ftp://... – JonMorehouse Nov 08 '13 at 20:08
  • 1
    pip uses requests which does not support FTP. http://stackoverflow.com/questions/15115328/python-requests-no-connection-adapters - use this url: http://xmlsoft.org/sources/python/libxml2-python-2.6.21.tar.gz – patroqueeet Jul 08 '14 at 12:01
  • And there are in this path only versions up to 2.6.21, from 2015, where libxml2 is now at 2.9.7 and of course the API changed so the compilation failed. – Patrick Mevzek May 18 '18 at 22:11
3

Before install lxml (on Debian):

apt-get install libxml2-dev libxslt1-dev pythonX.X-dev -y

Where pythonX.X is python2.7 or python2.6 or other needled python version.

After install system packages:

workon %environment_name%
pip install lxml --upgrade
Dunaevsky Maxim
  • 3,062
  • 4
  • 21
  • 26
  • I had to use `libxslt-dev` instead of `libxslt1-dev` Ubuntu 12.04 Python 2.7.3 – anglinb Dec 18 '14 at 21:06
  • Yes, this is trues also for Cygwin (Fedora), only the development headers are called `libxml2-devel` and `libxslt-devel`. And yes, use `lxml`, instead. – not2qubit Nov 08 '18 at 15:50
3

libxml2 is a C library not a python package so you cannot use Pip to install it. Normally it is already installed on almost every Linux distro out there. If you need to install it it's just

sudo apt-get install libxml2

If for some reason you absolutely need to have a local install you will need to grab and install the .deb package or the RPM. Barring that you can download the source and build it.

If you are fine with using the common copy but don't want to have /usr/local/ in your path, then just symlink it within your virtualenv.

You can find more info (than you probably wanted) at http://xmlsoft.org/

Scrapy lists it in their requirements:

  • Python 2.5 or Above
  • Twisted 2.5.0 or above
  • libxml2 2.6.28 or above (including Python bidings)
  • pyopenssl - only if you want to crawl secure (HTTPS) pages
zenWeasel
  • 1,889
  • 2
  • 22
  • 27
  • 1
    Pretty sure the OP was referring to the Python package called "libxml2", i.e. "import libxml2" not the C library (which the python package wraps). – Jason Antman Dec 18 '13 at 12:58
2

I have just come to this problem, with Ubuntu 14.04 kernel.

I already installed lxml using pip.

When I try to pip install lxml --upgrade inside the virtualenv, it always gave me a

x86_64-gnu-gcc exit 1

I solved this using sudo apt-get install libssl-dev.

davejal
  • 6,009
  • 10
  • 39
  • 82
shwsun
  • 27
  • 4
  • The reason that worked, is probably because that operation also installed the development libraries of `libxml2-dev` and `libxslt-dev`, which is the most common cause for pip install problems with `lxml`. – not2qubit Nov 08 '18 at 15:53
1

Alternatively, if you are on windows, as I suspect from your question, you need to either get the libxml2 binary-- there are links on the scrapy site, and as of Nov 2010, a version has been compiled that will work with everything-- or get the current trunk/dev version of scrapy, which works with lxml2. For virtualenv, since I'm not sure how to setup with an extra binary, the latter approach might be best. I've adopted the latter approach and it works flawlessly for me so far. Thanks to Pablo Hoffman the ultra-helpful creator of Scrapy (when I posted a question much like this one on Scrapy's mailing list, he released this change to trunk almost the next day). Note: libxml2 binary that worked with python 2.7 wasn't yet available at that time.

Profane
  • 1,128
  • 8
  • 13
0

If you installed python-libxml2 using apt-get (or other package manager) you can access it from virtualenv with the --system-site-packages flag.

When creating your virtualenv just specify this flag, e.g: 

virtualenv --system-site-packages env

This way your virtualenv will inherit the libxml2 package which is installed system wide.

not2qubit
  • 14,531
  • 8
  • 95
  • 135
ApriOri
  • 2,618
  • 29
  • 48