12

Using Python 2.7.3 with Numpy 1.6.2 on a 64-bit Ubuntu 12.04. Additional versions are present on the system (Python 2.6.4 and Numpy 1.6.1) but to the best of my knowledge these have no influence on the events described below.

I'm working on a Python program which uses Numpy, and getting an error when I try to run a certain command from the program. So, I decided to install python2.7-dbg to run the program with it and see if that can help with debugging. So I run

$ python-dbg <command>

instead of

$ python <command>

However, this gives an "undefined symbol: Py_InitModule4_64" error related to numpy. It doesn't matter what exactly I try to run; the error also occurs for commands that succeed if run with the "regular" python. It occurs even if I try to import numpy into a blank python-dbg prompt:

$ python-dbg
Python 2.7.3 (default, Aug  1 2012, 04:55:00)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 137, in <module>
    import add_newdocs
  File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 9, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 4, in <module>
    from type_check import *
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 8, in <module>
    import numpy.core.numeric as _nx
  File "/usr/local/lib/python2.7/dist-packages/numpy/core/__init__.py", line 5, in <module>
    import multiarray
ImportError: /usr/local/lib/python2.7/dist-packages/numpy/core/multiarray.so: undefined symbol: Py_InitModule4_64
[134187 refs]
>>>

Importing numpy succeeds when in a pure python prompt.

I've had no luck finding information on the internet. The closest match to my error is this bug on debian which is the exact same situation but has been closed before any further information was provided. I've installed both the python-numpy-dbg and the python-apt-dbg packages from APT; I have sudo access to the machine so the changes I make are valid system-wide. As far as I know, no chroot options have been set for any of the processes I'm trying to run, so the Debian bug page is of no help to me.

Any assistance will be greatly appreciated.

Boris
  • 1,180
  • 1
  • 18
  • 29
  • You might want to check your `$PYTHONPATH` environment variable. Just looking at the python-numpy-dbg package for Ubuntu it should not be importing from /usr/local/lib. That's what looks odd about this. – Iguananaut Nov 27 '12 at 15:20
  • Or, more to the point, even if there's nothing on `$PYTHONPATH` it looks like you installed your own copy of Numpy (maybe using pip or easy_install) and it's superseding the Numpy installed by apt. – Iguananaut Nov 27 '12 at 15:26
  • Thanks, I think you've set me on the right track. The previous version of numpy (1.6.1) which was present on the system since before I joined the project is in /usr/lib/python2.7/dist-packages, and the one I installed (1.6.2) via pip has been put in /usr/local/lib/python2.7/dist-packages. I didn't think this should matter too much but now that I checked sys.path, numpy wasn't there. So now I'll try to either add the new numpy to the path, or revert to the old one. – Boris Nov 27 '12 at 15:41
  • Right. So when I ran `pip uninstall numpy`, the 1.6.2 in /usr/local/lib was removed and the 1.6.1 in /usr/lib became "the" numpy. `import numpy` now works in a python-dbg prompt so I'll see if this suits me for the time being. Any idea how I can later make 1.6.2 in /usr/local/lib work with python-dbg? Adding it to the path didn't seem to work. – Boris Nov 27 '12 at 15:53
  • Since that seemed to work I'll go ahead and add it as an answer and then I'll address your other question. – Iguananaut Nov 27 '12 at 16:21

1 Answers1

4

As confirmed by the OP, the clue here is in the traceback which shows that numpy you're importing is in /usr/local/lib/. However, packages installed by apt go into /usr/lib/pythonX.Y, where as non-Ubuntu Python packages installed with pip, easy_install, etc. are installed under /usr/local/lib/pythonX.Y and supersedes the system packages.

For now you should uninstall the Numpy you have installed under /usr/local/lib/python2.7/dist-packages in order for the one installed by python-numpy to work. In the future you might be able to have both installed and do something with usercustomize.py to switch between them, but I'm not on an Ubuntu machine right now so I have no way of testing that out.

Iguananaut
  • 21,810
  • 5
  • 50
  • 63
  • Thanks a lot! I was afraid it will take hours or even days before someone replies but you answered soon enough for me to resolve this quickly and move on :) – Boris Nov 27 '12 at 16:38
  • 5
    Are you really saying that compiled python extensions installed via pip or easy_install (the standard for python modules) won't work with python-dbg? That sounds like a major bug to me. – papercrane Jun 28 '13 at 20:19
  • 3
    One cause of this error can also be that you run with site-packages (from a virtualenv) for a different python version (i.e. non dbg version). To fix setup a new virtualenv pointing to the right python binary and load packages from requirements file. – Paul Bormans Nov 04 '15 at 20:02