38

I've tried this and run in to problems a bunch of times in the past. Does anyone have a recipe for installing lxml on OS X without MacPorts or Fink that definitely works?

Preferably with complete 1-2-3 steps for downloading and building each of the dependencies.

syntagma
  • 23,346
  • 16
  • 78
  • 134
Simon Willison
  • 15,642
  • 5
  • 36
  • 44

15 Answers15

35

Thanks to @jessenoller on Twitter I have an answer that fits my needs - you can compile lxml with static dependencies, hence avoiding messing with the libxml2 that ships with OS X. Here's what worked for me:

cd /tmp
curl -O http://lxml.de/files/lxml-3.6.0.tgz
tar -xzvf lxml-3.6.0.tgz 
cd lxml-3.6.0
python setup.py build --static-deps --libxml2-version=2.7.3  --libxslt-version=1.1.24 
sudo python setup.py install
nyarasha
  • 1,119
  • 1
  • 14
  • 24
Simon Willison
  • 15,642
  • 5
  • 36
  • 44
  • 4
    The 'cd libs/' step fails for me, but it along with the following two wgets seem to be unnecessary as they happen as a side-effect of the 'setup.py build...' step anyway. – fuzzyman Aug 14 '09 at 17:31
  • Hmm... it's possible that libs/ directory was created for me the first time I ran "python setup.py build --static-deps" and it failed (because my firewall didn't allow FTP) - I actually pulled the .tar.gz files down via an intermediary server to work around that problem. – Simon Willison Aug 15 '09 at 07:19
  • 2
    AFAIK, wget doesn't come stock w/ OS X. Use `curl -O` instead. – David Eyk Aug 17 '09 at 18:54
  • This seems to be failing on Snow Leopard with the error: from SAX.c:15: /Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory – alxp Jan 30 '10 at 22:48
  • Ahh found the Snow Leopard solution here: http://blog.coredumped.org/2009/09/snow-leopard-and-lxml.html - must temporarily change /usr/bin/gcc to point to /usr/bin/gcc-4.0 – alxp Jan 30 '10 at 23:05
  • As of 04/06/2010, this solution fails when using the most recent versions of all 3 (lxml 2.2.6, libxml 2.7.7, libxslt 1.1.26). I get an error that starts with: Undefined symbols for architecture i386: "_gzdirect", referenced from: ___xmlParserInputBufferCreateFilename in libxml2.a(xmlIO.o) – andyashton Apr 07 '10 at 14:01
  • This problem has a very simple solution: http://stackoverflow.com/questions/1277124/how-do-you-install-lxml-on-os-x-leopard-without-using-macports-or-fink/6545556#6545556 – Andrei Jul 04 '11 at 11:05
  • The ftp doesn't work for me but I found the two dependent libraries here: libxml2-2.7.3.tar.gz at http://code.google.com/p/autosetup1/downloads/detail?name=libxml2-2.7.3.tar.gz&can=2&q= , libxslt-1.1.24.tar.gz http://code.google.com/p/savory/downloads/detail?name=libxslt-1.1.24.tar.gz&can=2&q= – I82Much Jul 23 '11 at 19:22
  • If you want to install lxml inside in virtualenv, activate virtualenv and run "pip install -e ." instead of "sudo python setup.py install" from last line. – bmihelac Jun 12 '12 at 10:53
33

This worked for me (10.6.8):

sudo env ARCHFLAGS="-arch i386 -arch x86_64" easy_install lxml
Andrei
  • 10,918
  • 12
  • 76
  • 110
  • I don't remember - but things to the effect of dylibs being of wrong architecture – I82Much Jul 26 '11 at 02:59
  • Hi, I ran this on 10.6.8 and it appears to have installed, but now when I try to import lxml in a script it still fails...any thoughts? If this is more approriate to move to it's own question I can do that. This is the end of the output after running the code above: `Installed /Library/Python/2.6/site-packages/lxml-2.3.3-py2.6-macosx-10.6-universal.egg Processing dependencies for lxml Finished processing dependencies for lxml` and this is what happens when I try to import: `ImportError: No module named lxml` – Stin Mar 24 '12 at 19:36
  • I put up a better formatted question [here](http://stackoverflow.com/questions/9854969/import-lxml-fails-on-osx-after-seemingly-successful-install) – Stin Mar 24 '12 at 19:50
  • @Jay have you tried what `chuckles` suggests: http://stackoverflow.com/a/7850559/179581 ? – Andrei Aug 10 '12 at 09:42
  • For me on OS X 10.6.8 it worked to install, but then the installation was broken. `import lxml` works, but `import lxml.etree` does not. Related: http://stackoverflow.com/questions/19557304/installing-lxml-on-mac-os-x-10-6-8-with-gcc-4-2 and http://stackoverflow.com/questions/17766725/how-to-re-install-lxml/19613627#19613627 – Sergey Orshanskiy Oct 27 '13 at 01:21
  • `sudo env ARCHFLAGS="-arch x86_64" easy_install-2.7 lxml` worked on Mac OS 10.9. Make sure to qualify the version of `easy_install` you are using. – maxko87 Jul 01 '14 at 19:03
  • @SergeyOrshanskiy did you ever find a working solution around the lxml.etree extension? – ganicus Jul 02 '14 at 22:45
  • @ganicus, no, I concluded that if you want to use Apple's products, you have to prepare to always upgrade everything to the newest hardware and software. Happily using Ubuntu... – Sergey Orshanskiy Jul 04 '14 at 07:37
  • This is sadly true. I did however have success with home brew and some doctored up installs as documented below. I'm working on a dated snow leopard server trying to automate some things with Python. Hopefully it will help anyone who was in our same spot. – ganicus Jul 04 '14 at 17:10
23

I've had excellent luck with Homebrew to install the libxml2 dependency:

brew install libxml2

Homebrew doesn't seem to have libxslt available, but I've not yet had a need for XSLT. YMMV.

Once you have the dependency(s), then the usual methods work just fine:

pip install lxml

or

easy_install lxml
Glorfindel
  • 21,988
  • 13
  • 81
  • 109
David Eyk
  • 12,171
  • 11
  • 63
  • 103
  • Doesn't work for me. I end up with `error: command 'gcc-4.2' failed with exit status 1`. The solution above does work, however. – Weston Ruter Apr 26 '11 at 04:10
  • Didn't work for me on 10.6, but I managed to install with http://stackoverflow.com/questions/1277124/how-do-you-install-lxml-on-os-x-leopard-without-using-macports-or-fink/6545556#6545556 – Andrei Jul 01 '11 at 08:41
  • 3
    on 10.11: error: command 'clang' failed with exit status 1; solution was: xcode-select --install – Josep Valls Oct 14 '15 at 15:25
  • Just leaving this here: I needed to run: `brew install libxml2` followed by `easy_install lxml` followed by `easy_install scour` for saving Inkscape files. – Prince Jul 19 '17 at 05:07
5

Easy_install can work using this:

STATIC_DEPS=true easy_install 'lxml>=2.2beta4'

you may then need to run, depending on permissions;

STATIC_DEPS=true sudo easy_install 'lxml>=2.2beta4'

see http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/

  • 1
    This solution didn't work for me (OSX 10.6.8), but I've found another one: http://stackoverflow.com/questions/1277124/how-do-you-install-lxml-on-os-x-leopard-without-using-macports-or-fink/6545556#6545556 – Andrei Jul 04 '11 at 11:07
5

using homebrew (0.9.5) on el capitan (10.11.1) the following worked for me:

brew install libxml2
LD_FLAGS=-L/usr/local/opt/libxml2/lib CPPFLAGS=-I/usr/local/opt/libxml2/include/libxml2 pip install lxml
mizerlou
  • 156
  • 1
  • 6
3

This worked for me on 10.8.5

  1. Install Xcode from Mac App Store
  2. Xcode -> Preferences -> Downloads -> Command Line Tools
  3. Install homebrew using
  4. ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"
  5. brew install libxml2
  6. sudo easy_install lxml

This comprises suggestions from:

But I wanted to compile it into one answer rather than leave comments everywhere

Community
  • 1
  • 1
William Entriken
  • 37,208
  • 23
  • 149
  • 195
  • Thank you, I was missing the part where installing the Xcode Command Line Tools was needed. This can be done via command line: `xcode-select --install` – nyarasha May 03 '16 at 19:39
3

On OS X 10.9.1 the suggested answer above errors out during install -- following changes had to be made:

cd /tmp  
curl -o lxml-3.3.0.tgz http://lxml.de/files/lxml-3.3.0.tgz  
tar -xzvf lxml-3.3.0.tgz  
cd lxml-3.3.0  
python setup.py build --static-deps --libxml2-version=2.8.0  --libxslt-version=1.1.24  
sudo python setup.py install  
user785278
  • 31
  • 1
  • 1
    On 10.9.2, this fails with: clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future] – EvanK Mar 23 '14 at 17:18
1

I'm using OSX 10.11 El Capitan and Homebrew. Using pip install lxml would give me "fatal error: 'libxml/xmlversion.h' file not found" and "failed with error code 1" blah blah.

According to the official website, I should use STATIC_DEPS=true pip install lxml (add sudo before pip if you need that), and that solved my problem.

I ran brew install libxml2 and brew install libxslt to install the dependencies while troubleshooting. I'm not sure if those two commands are necessary.

feilong
  • 649
  • 5
  • 7
1

I had this working fine with Snow Lepoard but after I upgraded to Lion I had to symlink gcc-4.2 to gcc. Running sudo env ARCHFLAGS="-arch i386 -arch x86_64" easy_install lxml was looking for gcc-4.2 instead of gcc.

chuckles
  • 226
  • 1
  • 7
0

To install with up to date versions of libxml2 and libxslt:

ARCHFLAGS="-arch i386 -arch x86_64" STATIC_DEPS=true pip install lxml

To install with specific versions of libraries:

ARCHFLAGS="-arch i386 -arch x86_64" STATIC_DEPS=true LIBXML2_VERSION=2.7.3 LIBXSLT_VERSION=1.1.24 pip install lxml

CentOS 64 bit (a bit off question, but hard won):

CFLAGS=-fPIC STATIC_DEPS=true pip install lxml

or

CFLAGS=-fPIC STATIC_DEPS=true LIBXML2_VERSION=2.7.3 LIBXSLT_VERSION=1.1.24 pip install lxml
m.kocikowski
  • 5,422
  • 2
  • 23
  • 9
0

Try installing Cython and installing from source, easy_install does fail. I haven't tried on my mac yet though.

Failing that the ports version isn't that ancient. You can see the dependencies, some of which had to be updated for my Linux build of lxml.

info py25-lxml py25-lxml @2.1.5 (python, devel)

lxml is a Pythonic binding for the libxml2 and libxslt libraries. It is unique in that it combines the speed and feature completeness of these libraries with the simplicity of a native Python API, mostly compatible but superior to the well-known ElementTree API. Homepage: http://codespeak.net/lxml/

Library Dependencies: python25, libxml2, libxslt, py25-hashlib, py25-setuptools, py25-zlib Platforms: darwin Maintainers: akitada@macports.org openmaintainer@macports.org

Nick Martin
  • 172
  • 3
  • Check out the post from Ross, you need the header files for libxml2 and libxslt. Think that's why GCC was barfing when I tried to compile from the svn trunk. – Nick Martin Aug 14 '09 at 11:34
0

This is quite up to date - march 2009: http://lsimons.wordpress.com/2008/08/31/how-to-install-lxml-python-module-on-mac-os-105-leopard/

Ross
  • 17,861
  • 2
  • 55
  • 73
0

lxml is included in the pypm repository:

$ pypm install lxml
Sridhar Ratnakumar
  • 81,433
  • 63
  • 146
  • 187
0

A lot of pain went into this for an outdated 10.6.8 os x but here it goes for anyone running Snow Leopard!

First you have to install a different version of libxml2 from homebrew and install --with-python. You can do this by typing in the following commands.

brew update
brew edit libxml2

Then find the line that says "--without-python" and change to "--with-python".

system "./configure", "--disable-dependency-tracking",
                      "--prefix=#{prefix}",
                      "--with-python"

Now you can install libxml2.

brew install libxml2

Next check your new install of libxml2 in the default homebrew location. You want to find the libxml2 config.

YOURS MAY BE DIFFERENT:

"/usr/local/Cellar/libxml2/VERSION_/bin/xml2-config"

Now use the following command to install lxml with pip using the newly installed libxml2 config and not the Mac OS X version.

ARCHFLAGS="-arch i386 -arch x86_64" pip install lxml --install-option="--with-xml2-config=/usr/local/Cellar/libxml2/2.9.1/bin/xml2-config"

Worked for me on my 10.6.8 Python 2.6. Thanks.

Credit goes to this page for showing me pip --install-option ...

http://natanyellin.com/page/4/

ganicus
  • 321
  • 5
  • 12
-5

I compile it in /usr/local without any issues whatsoever.

Install Python, libxml2, libxslt and then lxml. You might need setuptools installed too.

  • That works on OS X? What prefix commands etc do you use? How do you ensure the libxml2 you are installing doesn't conflict with the one that ships with OS X? – Simon Willison Aug 14 '09 at 11:01