4

I'm trying to install numpy==1.10.2, scipy and matplotlib on server based on opensuse. I've installed numpy in virtualenv from source (I've tried by pip also - same result of course). Now when I'm trying to import numpy in python console I'm receiving following error:

ImportError: /home/user/.virtualenvs/project/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: cblas_sgemm

Note: I'm not superuser on this server.

Edit:

ldd /home/user/.virtualenvs/project/lib/python2.7/site-packages/numpy/core/multiarray.so`
linux-vdso.so.1 (0x00007fffa0d69000)
libtatlas.so.3 => /home/user/.local/usr/lib64/atlas/libtatlas.so.3 (0x00007fe366d66000)
libm.so.6 => /lib64/libm.so.6 (0x00007fe366a50000)
libpython2.7.so.1.0 => /usr/lib64/libpython2.7.so.1.0 (0x00007fe3666b2000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe366496000)
libc.so.6 => /lib64/libc.so.6 (0x00007fe3660f0000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe367a15000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fe365eec000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fe365ce9000)

libtatlas.so.3 => /home/user/.local/usr/lib64/atlas/libtatlas.so.3 - I linked this because of other problem with libatlas: ImportError: /usr/lib64/atlas/libtatlas.so.3: undefined symbol: clapack_ilaenv

SOLLUTION

The problem was with ATLAS lib. According to @ali_m advice I installed numpy from source with OpenBLAS instead of ATLAS. Here link to instruction how to install numpy with BLAS.

Community
  • 1
  • 1
Darkowic
  • 661
  • 8
  • 17
  • What BLAS library are you linked against? Could you show the output of `$ ldd /home/user/.virtualenvs/project/lib/python2.7/site-packages/numpy/core/multiarray.so`? – ali_m Jan 05 '16 at 19:39
  • I've edited original post with output from ldd – Darkowic Jan 05 '16 at 19:48
  • Hmm... Did you compile that ATLAS library yourself? IIRC, building ATLAS is quite a fiddly process, so I suppose it's possible that something went wrong. I would recommend using OpenBLAS instead - it's much easier to compile, and is consistently faster than ATLAS in every benchmark I've come across so far. – ali_m Jan 05 '16 at 20:32
  • Ok. ATLAS compilation looks pretty hard. How should I use BLAS with numpy? I don't know anything about this libraries... – Darkowic Jan 05 '16 at 21:29
  • Well, where did that ATLAS library in `/home/user/.local/usr/lib64/atlas/libtatlas.so.3` come from if you didn't build it yourself? – ali_m Jan 05 '16 at 21:46
  • You could look at my answer [here](http://stackoverflow.com/a/14391693/1461210) if you just want instructions for building numpy and OpenBLAS – ali_m Jan 05 '16 at 21:47
  • 1
    I unpacked this lib from rpm package... A little workaround. I will try to compile it tomorrow. Thank you very much for help :) – Darkowic Jan 05 '16 at 21:52
  • On the version of ATLAS installed on my Ubuntu machine the `clapack_` and `cblas_` symbols are defined in another library called `liblapack_atlas.so`. That might come from another RPM package (if you were able to install using your system package manager then it ought to sort out these dependencies for you automatically). Anyway, I would recommend OpenBLAS over ATLAS. – ali_m Jan 05 '16 at 21:58
  • You rock @ali_m ! Your advice about installing numpy with BLAS helped me and now everything works well :) Thank you! – Darkowic Jan 06 '16 at 13:07

1 Answers1

5

As you mentioned in the comments above, you extracted libtatlas.so.3 from an RPM package and copied it into a local directory, rather than either building it from source or installing it via the package manager. I'm sure that this is the cause of the problem.

In particular, the clapack_* and cblas_* symbols don't seem to be defined within libtatlas.so.3. On my Ubuntu machine, if I sudo apt-get install libatlas-base-dev I get two different shared libraries:

~$ ldconfig -p | grep atlas
        liblapack_atlas.so.3 (libc6,x86-64) => /usr/lib/liblapack_atlas.so.3
        liblapack_atlas.so (libc6,x86-64) => /usr/lib/liblapack_atlas.so
        libatlas.so.3 (libc6,x86-64) => /usr/lib/libatlas.so.3
        libatlas.so (libc6,x86-64) => /usr/lib/libatlas.so

The clapack_* and cgemm_* symbols are defined in liblapack_atlas.so.3 rather than libatlas_so.3:

~$ nm -D /usr/lib/libatlas.so.3 | grep clapack_ilaenv

~$ nm -D /usr/lib/libatlas.so.3 | grep cblas_sgemm

~$ nm -D /usr/lib/liblapack_atlas.so.3 | grep clapack_ilaenv
0000000000041d90 T clapack_ilaenv

~$ nm -D /usr/lib/liblapack_atlas.so.3 | grep cblas_sgemm
                 U cblas_sgemm

Given that you have no admin rights, and therefore can't install ATLAS via the normal system package manager, here are three potential solutions:

  • It might be possible to extract a binary of liblapack_atlas.so.3 (or whatever the OpenSUSE equivalent is) from a different RPM package, and get numpy to link against it by modifying the site.cfg file in the numpy source directory.
  • Another option would be to build ATLAS from source, but in my experience this is a long and painful process.
  • My personal recommendation would be to build OpenBLAS instead, as described in my previous answer here. It's much easier to compile, and has better performance in every benchmark that I've come across so far.
Community
  • 1
  • 1
ali_m
  • 71,714
  • 23
  • 223
  • 298