6

When I use any import statement within modules I compile with cython, I receive the following error on importing the modules (see full code below):

ImportError: /.../hw.cpython-35m-x86_64-linux-gnu.so: undefined symbol: __intel_sse2_strchr

Everything works fine on my own machine, but I get this error when trying to recompile and run my scripts on an external high-performance computer.

Googling I saw similar errors pop up in several places, but in these discussions the problem is either related to an error in the own code, rather than an imported module (e.g. Undefined symbol error importing Cython module) or something goes wrong at building cython or compile time (e.g. Can cython be compiled with icc?). I don't see how to apply these to my case.

From reading other discussions, I suspected the problem was in the formulation of my setup.py, but I really don't know much about that - on my own system the standard examples simply worked. I tried adding numpy to the setup (see setup2.py below), but that didn't solve the problem - I get the same error on import hw.

Full example

hw.pyx:

import numpy  # without this line, the example works fine 

cpdef testfun():
    print("Hello world!")

setup.py:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize('hw.pyx'),
)

setup2.py:

from distutils.core import setup
from Cython.Build import cythonize
import numpy as np

setup(
    ext_modules = cythonize('hw.pyx'),
    include_dirs = [np.get_include()]  # -> including this doesn't make a difference
)

System info

The external system runs python 3.5.2. I run my code within a virtualenv, where I have Cython 0.28.1. I tried both numpy 1.13.0 and 1.14.2. For the virtuel environment in this example, I did not install any other packages (probably not related to my problem, but pip freeze also lists UNKNOWN==0.0.0)

[IMPORTANT EDIT]

I received the error for numpy (and since I had struggled before with cdef np.ndarray I figured that was the problem) - but actually it is any import statement that causes the error - numpy just happened to be the first module that was imported. Still no clue how to resolve this though.

Scipio
  • 313
  • 2
  • 15
  • 1
    It looks as if you would compile with intel compiler, but don’t provide intel runtime libraries to the linker. Switch to gcc or provide the needed runtime libraries – ead Mar 28 '18 at 04:40
  • Thanks, using `CC=gcc python setup.py build_ext --inplace` gives a working module! Is there a speed advantage in using the intel compiler? How can I find out what the needed runtime libraries are, and how would I make those available? – Scipio Mar 28 '18 at 09:34
  • Ah I probably should use a different Python distribution that is also available: `python/3.5.2-intel-u2`. If I try this, I get a different undefined symbol error on import: `undefined symbol: __stack_chk_guard` – Scipio Mar 28 '18 at 09:45
  • There seems to be a problem with loading the intel Python and the `mpi` module at the same time, whereas I need the `mpi` module as well. So I will just go with the `gcc` compiler – Scipio Mar 28 '18 at 09:50

1 Answers1

1

The problem was that I used the Intel compiler without running the Intel Python distribution, or otherwise providing the Intel runtime libraries.

Thanks to @ead, I resolved the problem by switching to GCC using:

CC=gcc python setup.py build_ext --inplace
Scipio
  • 313
  • 2
  • 15