Installing/loading github::KMeansRex Python library

Question

I am a python noob, trying to run the example in the KmeansRex library from github. I use a ubuntu machine. This is a library that doesn't have a setup.py file. Despite several attempts, I have not been able to run the short code example shown on the project's github page's README file.

To try to install this library, I do:

git clone https://github.com/michaelchughes/KMeansRex.git

Then I do (as per the instructions on the github page, just naming the .so file libkmeansrex64.so instead of libkmeansrex.so as I am on a 64bit machine):

g++ --shared -o libkmeansrex64.so KMeansRexCore.cpp -I/home/path/to/eigen/ -O3 -DNDEBUG

/usr/bin/ld: /tmp/ccdmUbg9.o: relocation R_X86_64_32S against `.bss' can not be used when making a shared object; recompile with -fPIC
/tmp/ccdmUbg9.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status

so I do:

g++ --shared -o libkmeansrex64.so KMeansRexCore.cpp -I/home/path/to/eigen/ -O3 -fpic -DNDEBUG

(and this compiles without errors). Then I do:

cd KMeansRex/

Then I do:

python

I follow the instructions on the github for the example:

Python 2.7.11+ (default, Apr 17 2016, 14:00:29) 
[GCC 5.3.1 20160413] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np;

(so far so good). But then, I do:

import KMeansRex

only to get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "KMeansRex/__init__.py", line 1, in <module>
    from KMeansRex import RunKMeans
  File "KMeansRex/KMeansRex.py", line 30, in <module>
    lib = ctypes.cdll.LoadLibrary( os.path.join(parentdir,'libkmeansrex64.so') )
  File "/usr/lib/python2.7/ctypes/__init__.py", line 443, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libkmeansrex64.so: cannot open shared object file: No such file or directory

Which is odd, because the compiler did produce libkmeansrex64.so:

/KMeansRex$ ls -n
total 84
drwxrwxr-x 2 1000 1000  4096 Mai 22 13:24 demonumpyctypes
drwxrwxr-x 3 1000 1000  4096 Mai 22 14:00 KMeansRex
-rw-rw-r-- 1 1000 1000  7003 Mai 22 13:24 KMeansRexCore.cpp
-rw-rw-r-- 1 1000 1000   635 Mai 22 13:24 KMeansRexCore.h
-rw-rw-r-- 1 1000 1000  2214 Mai 22 13:24 KMeansRex.cpp
-rwxrwxr-x 1 1000 1000 41792 Mai 22 13:24 libkmeansrex64.so
-rw-rw-r-- 1 1000 1000  1541 Mai 22 13:24 LICENSE
-rw-rw-r-- 1 1000 1000  7899 Mai 22 13:24 mersenneTwister2002.c
-rw-rw-r-- 1 1000 1000  1396 Mai 22 13:24 README

Edit:

Problem solved (thanks to user kvorobiev). So before the line

  lib = ctypes.cdll.LoadLibrary( os.path.join(parentdir,'libkmeansrex64.so') )

I added

  print os.path.join(parentdir,'libkmeansrex64.so')

and the prompt prints:

libkmeansrex64.so

so it is clear that I should replace

  lib = ctypes.cdll.LoadLibrary( os.path.join(parentdir,'libkmeansrex64.so') )

By:

lib = ctypes.cdll.LoadLibrary( os.path.join(parentdir,'/path/to/libkmeansrex64.so') )

Doing this, it all works fine;)

@renemilk: soory I do not understand. Could you explain better how to change the `dlopen` call? — user189035, May 22 '16 at 12:25
Just open `KMeansRex.py` file and insert `print os.path.join(parentdir,'libkmeansrex64.so')` line just before `lib = ctypes.cdll.LoadLibrary( os.path.join(parentdir,'libkmeansrex64.so') )` and add results to your question — kvorobiev, May 22 '16 at 12:27
@kvorobiev: that fixed the problem! Should I delete this question? — user189035, May 22 '16 at 12:32
I mean I replaced ` lib = ctypes.cdll.LoadLibrary( os.path.join(parentdir,'libkmeansrex64.so') )` by ` lib = ctypes.cdll.LoadLibrary( os.path.join(parentdir,'/path/to/libkmeansrex64.so') )` — user189035, May 22 '16 at 12:33
I added a full answer with explanation. If it is helpful you could mark it as right answer) — kvorobiev, May 22 '16 at 13:22

score 2 · Accepted Answer · edited May 23 '17 at 12:31

There is a problem with paths in KMeansRex.py.

import os
...

curdir = os.path.split( __file__ )[0]
parentdir = os.path.split( curdir)[0]
...

In Python3 this code will work correctly, since __file__ contains absolute path to file. But in Python2 if you aren't inside the part of sys.path that contains the module, you'll get an absolute path. If you are inside the part of sys.path that contains the module, you'll get a relative path.
You could use

import os
...

curdir = os.path.split( os.path.abspath(__file__) )[0]
parentdir = os.path.split( curdir)[0]
...

or just hardcode absolute path to your library

lib = ctypes.cdll.LoadLibrary( '/abs/path/to/lib/libkmeansrex64.so' )

Installing/loading github::KMeansRex Python library

Edit:

1 Answers1