Installing lxml with pip in virtualenv Ubuntu 12.10 error: command 'gcc' failed with exit status 4

Question

I'm having the following error when trying to run "pip install lxml" into a virtualenv in Ubuntu 12.10 x64. I have Python 2.7.

I have seen other related questions here about the same problem and tried installing python-dev, libxml2-dev and libxslt1-dev.

Please take a look of the traceback from the moment I tip the command to the moment when the error occurs.

Downloading/unpacking lxml
  Running setup.py egg_info for package lxml
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.1.2.
    Building without Cython.
    Using build configuration of libxslt 1.1.26
    Building against libxml2/libxslt in the following directory: /usr/lib

    warning: no files found matching '*.txt' under directory 'src/lxml/tests'
Installing collected packages: lxml
  Running setup.py install for lxml
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.1.2.
    Building without Cython.
    Using build configuration of libxslt 1.1.26
    Building against libxml2/libxslt in the following directory: /usr/lib
    building 'lxml.etree' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/home/admin/.virtualenvs/dev.actualito.com/build/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o
    src/lxml/lxml.etree.c: In function '__pyx_f_4lxml_5etree__getFilenameForFile':
    src/lxml/lxml.etree.c:26851:7: warning: variable '__pyx_clineno' set but not used [-Wunused-but-set-variable]
    src/lxml/lxml.etree.c:26850:15: warning: variable '__pyx_filename' set but not used [-Wunused-but-set-variable]
    src/lxml/lxml.etree.c:26849:7: warning: variable '__pyx_lineno' set but not used [-Wunused-but-set-variable]
    src/lxml/lxml.etree.c: In function '__pyx_pf_4lxml_5etree_4XSLT_18__call__':
    src/lxml/lxml.etree.c:138273:81: warning: passing argument 1 of '__pyx_f_4lxml_5etree_12_XSLTContext__copy' from incompatible pointer type [enabled by default]
    src/lxml/lxml.etree.c:136229:52: note: expected 'struct __pyx_obj_4lxml_5etree__XSLTContext *' but argument is of type 'struct __pyx_obj_4lxml_5etree__BaseContext *'
    src/lxml/lxml.etree.c: In function '__pyx_f_4lxml_5etree__copyXSLT':
    src/lxml/lxml.etree.c:139667:79: warning: passing argument 1 of '__pyx_f_4lxml_5etree_12_XSLTContext__copy' from incompatible pointer type [enabled by default]
    src/lxml/lxml.etree.c:136229:52: note: expected 'struct __pyx_obj_4lxml_5etree__XSLTContext *' but argument is of type 'struct __pyx_obj_4lxml_5etree__BaseContext *'
    src/lxml/lxml.etree.c: At top level:
    src/lxml/lxml.etree.c:12384:13: warning: '__pyx_f_4lxml_5etree_displayNode' defined but not used [-Wunused-function]
    gcc: internal compiler error: Killed (program cc1)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See  for instructions.
    error: command 'gcc' failed with exit status 4
    Complete output from command /home/admin/.virtualenvs/dev.actualito.com/bin/python -c "import setuptools;__file__='/home/admin/.virtualenvs/dev.actualito.com/build/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-asDtN5-record/install-record.txt --single-version-externally-managed --install-headers /home/admin/.virtualenvs/dev.actualito.com/include/site/python2.7:
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'

  warnings.warn(msg)

Building lxml version 3.1.2.

Building without Cython.

Using build configuration of libxslt 1.1.26

Building against libxml2/libxslt in the following directory: /usr/lib

running install

running build

running build_py

copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.7/lxml/includes

running build_ext

building 'lxml.etree' extension

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/home/admin/.virtualenvs/dev.actualito.com/build/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o

src/lxml/lxml.etree.c: In function '__pyx_f_4lxml_5etree__getFilenameForFile':

src/lxml/lxml.etree.c:26851:7: warning: variable '__pyx_clineno' set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c:26850:15: warning: variable '__pyx_filename' set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c:26849:7: warning: variable '__pyx_lineno' set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c: In function '__pyx_pf_4lxml_5etree_4XSLT_18__call__':

src/lxml/lxml.etree.c:138273:81: warning: passing argument 1 of '__pyx_f_4lxml_5etree_12_XSLTContext__copy' from incompatible pointer type [enabled by default]

src/lxml/lxml.etree.c:136229:52: note: expected 'struct __pyx_obj_4lxml_5etree__XSLTContext *' but argument is of type 'struct __pyx_obj_4lxml_5etree__BaseContext *'

src/lxml/lxml.etree.c: In function '__pyx_f_4lxml_5etree__copyXSLT':

src/lxml/lxml.etree.c:139667:79: warning: passing argument 1 of '__pyx_f_4lxml_5etree_12_XSLTContext__copy' from incompatible pointer type [enabled by default]

src/lxml/lxml.etree.c:136229:52: note: expected 'struct __pyx_obj_4lxml_5etree__XSLTContext *' but argument is of type 'struct __pyx_obj_4lxml_5etree__BaseContext *'

src/lxml/lxml.etree.c: At top level:

src/lxml/lxml.etree.c:12384:13: warning: '__pyx_f_4lxml_5etree_displayNode' defined but not used [-Wunused-function]

gcc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See  for instructions.

error: command 'gcc' failed with exit status 4

----------------------------------------
Command /home/admin/.virtualenvs/dev.actualito.com/bin/python -c "import setuptools;__file__='/home/admin/.virtualenvs/dev.actualito.com/build/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-asDtN5-record/install-record.txt --single-version-externally-managed --install-headers /home/admin/.virtualenvs/dev.actualito.com/include/site/python2.7 failed with error code 1 in /home/admin/.virtualenvs/dev.actualito.com/build/lxml
Storing complete log in /home/admin/.pip/pip.log

Yes, I have just run this command: sudo aptitude install build-essential, is there a problem to do it after the other things? — Cristian Rojas, Apr 22 '13 at 14:32
Easy to check - `pip uninstall lxml` then `pip install lxml` to see if the result is different. I think it will be. — cms_mgr, Apr 22 '13 at 14:35
I have reinstalled python-dev, libxml2-dev and libxslt1-dev libraries with sudo and apt-get remove --purge, tried again to pip install lxml into the virtualenv but the error code is the same as before. — Cristian Rojas, Apr 22 '13 at 14:43
In my case, I need libxslt1-dev. Use this command : sudo apt-get install libxml2-dev libxslt1-dev python-dev — KyungHoon Kim, May 31 '14 at 06:57
For not debian users (i.e. Red Hat, CentOS) yum install python-devel, libxml2-devel, libxslt-devel ... the 'dev' packages become 'devel' — Nabeel Ahmed, Oct 14 '15 at 10:07

score 73 · Answer 1 · answered Jun 14 '13 at 13:59

73

Make sure you have enough memory. Try dmesg | tail to see if it outputs something like:

...
[3778136.277570] Out of memory: Kill process 21267 (cc1) score 557 or sacrifice child
[3778136.277587] Killed process 21267 (cc1) total-vm:365836kB, anon-rss:336228kB, file-rss:0kB

answered Jun 14 '13 at 13:59

Matmas

1,570
1
14
10

2

This happened to me. What do I do? I dont think I can get more memory - it's a VPS. – SiddharthaRT Jun 19 '13 at 09:55
5

@SiddharthaRT: as a workaround you could install `python-lxml` package from official ubuntu repositories and have your virtualenv created with the `--system-site-packages` switch. – Matmas Jun 20 '13 at 17:29
16

[this so post might help if you need more memory](http://stackoverflow.com/questions/18334366/out-of-memory-issue-in-installing-packages-on-ubuntu-server) – Scott Coates Oct 19 '13 at 20:18
1

You can add swap memory on file. See this [link](https://www.digitalocean.com/community/articles/how-to-add-swap-on-ubuntu-14-04) – Thales Ceolin May 26 '14 at 19:58
I had the same problem with *lxml* building a docker container on *AWS EB*. **One more solution**: do not build in the cloud, build local and push to a repository (could be *Docker Hub*, *AWS Registry*, ...) and use already baked image. – hsrv Feb 10 '16 at 18:08
I found information on making a very large swapfile in the answer [here](https://www.digitalocean.com/community/questions/ubuntu-2gb-swap-file), and information on temporarily resizing (which I did not ultimately have to do) [here](https://www.digitalocean.com/community/tutorials/how-to-resize-your-droplets-on-digitalocean). Thanks! – Max von Hippel Nov 28 '16 at 05:19

score 25 · Accepted Answer · answered Apr 22 '13 at 15:14

25

Here is the my saved note.

sudo apt-get install libxml2
sudo apt-get install libxslt1.1
sudo apt-get install libxml2-dev
sudo apt-get install libxslt1-dev
sudo apt-get install python-libxml2
sudo apt-get install python-libxslt1
sudo apt-get install python-dev
sudo apt-get install python-setuptools

easy_install lxml

It has worked for my ubuntu 12.10

answered Apr 22 '13 at 15:14

Delgermurun

648
9
19

1

It didn't work initially, but I circumvented the problem. I've discovered that, for some reason, it is picking easy_install from system installed packages. So, if I run easy_install explicitly from my version of Python compiled from sources, it works :) For example: "/opt/python/2.7.5/bin/easy_install lxml" – Richard Gomes Oct 21 '13 at 15:51
easy_install vs pip install did the trick, for my ubuntu 13.10 & python 2.7. Thanks! – haltabush Nov 14 '13 at 17:53
I've stumbled with the same problem again. This time using Python3. Then I've decided to write a full answer about this. – Richard Gomes Nov 25 '13 at 17:13
I found the following enough: `apt-get install libxml2-dev libxslt1-dev python-dev` – Kamil Sindi Aug 24 '15 at 02:04
done everything still fails with this message : `/usr/bin/ld: cannot find -lz` , please help! – Koustuv Sinha Feb 21 '16 at 15:35
1

@KoustuvSinha: This solved my problem: http://stackoverflow.com/questions/3373995/usr-bin-ld-cannot-find-lz – HereHere Feb 22 '16 at 16:29
thank you @HereHere! i had already removed lxml dependency and proceeded with an alternative, but thanks anyway, would be handy later – Koustuv Sinha Feb 23 '16 at 13:10

score 18 · Answer 3 · edited Feb 23 '17 at 19:58

18

According to lxml site you could use such construction:

CFLAGS="-O0"  pip install lxml

installation guide

Note for those installing globally: The proper way to pass environment variables with sudo is after sudo:

sudo CFLAGS="-O0" pip install lxml

edited Feb 23 '17 at 19:58

Chris W.

37,583
36
99
136

answered Apr 07 '14 at 12:20

WhiteDragon

468
6
13

This worked for me on Ubuntu 14.04 like a charm even with low ram (512MB), thanks. – Mansoor Akram Sep 30 '15 at 09:15
Worked for me, didn't see this answer before posting mine – Ignacio Vazquez Mar 15 '16 at 18:19
Worked on `ubuntu 16.04` – tread Mar 21 '17 at 13:56

score 8 · Answer 4 · answered Jul 03 '14 at 07:21

8

I met the similar question(error: command 'gcc' failed with exit status 4) this morning. It seems you need check your machine's memory. If the memory is lower than 512M,that may be the cause.Try to close some services temporarily,like apache server,and try "pip install lxml" again.It maybe work!

answered Jul 03 '14 at 07:21

xmduhan

965
12
14

Increasing the memory did the trick! I was deploying a Django application in a DigitalOcean dokku droplet with 512MB RAM and was getting `error: command 'gcc' failed with exit status 4`. Resizing the droplet to have 1GB RAM solved the problem. – silentser Aug 01 '14 at 21:32

Richard Gomes · Answer 5 · 2013-11-25T17:34:36.907

I've stumbled with this trouble a couple of times.

Short answer

Python2: $ python2.7 setup.py clean build --with-cython install
Python3: $ pip-3.3 install lxml

Long answer

The hypothesis is that pip install lxml should work in every environment, regardless if you are using Python2 or Python3.

There's also Cython to be considered: You will certainly enjoy lxml compiled with Cython due to relevant performance gains.

For reasons unknown to me, the compilation on Python2 does not find Cython. To be more precise and absolutely explicit about this matter, both commands below DO NOT employ Cython:

# DO NOT use these commands. I repeat: DO NOT use these commands.
$ pip-2.7 install lxml
$ easy_install-2.7 install lxml

So, when using Python2 you have only one alternative, as far as I know, which is: compile from sources, Luke!

# install build environment and dependencies
$ kernel_release=$( uname -r )
$ sudo apt-get install linux-headers-${kernel_release} build-essential -y
$ sudo apt-get install libxml2-dev libxslt1-dev -y

# Download from github and compile from sources
$ git clone --branch lxml-3.2.4 https://github.com/lxml/lxml
$ python2.7 setup.py clean build --with-cython install

score 2 · Answer 6 · answered Aug 15 '15 at 11:48

2

For ubuntu 12.04 and virtual env:

sudo apt-get install libxml2-dev libxslt-dev
workon some-virt-env
pip install lxml

answered Aug 15 '15 at 11:48

Kostyantyn

5,041
3
34
30

score 2 · Answer 7 · answered Mar 15 '16 at 17:37

2

Try to disable the C compiler optimizations by setting the FLAGS environment variable

CFLAGS="-O0"  pip install lxml

That solves for me without the need of more RAM

answered Mar 15 '16 at 17:37

Ignacio Vazquez

534
5
9

1

Thanks! Your advice helps me to solve issues on AWS micro instance. – Alex G.P. Aug 02 '16 at 14:48

Installing lxml with pip in virtualenv Ubuntu 12.10 error: command 'gcc' failed with exit status 4

7 Answers7

Linked

Related