0

I have a somewhat strange setup: I have a libmylib.so I cannot modify, but, since the interface is rather convoluted, I wrote a simple wrapper directly into ffbuilder.set_source()

My full lib_builder.py is:

import os
from cffi import FFI
ffibuilder = FFI()

ffibuilder.cdef("""
int start(int port);
int ready(void);
int publish(char *buf, int len);
int stop(void);
""")

ffibuilder.set_source('_mylib_cffi', """
#include "mylib.h"

static uint32_t handle = 0xffffffff;
int start(int port) {
    mylib_init(0);
    mylib_register_publisher("127.0.0.1", port, &handle, 0);
    return 1;
}
int ready(void) {
    return handle != 0xffffffff;
}
int publish(char *buf, int len) {
    return mylib_publish(handle, buf, len);
}
int stop(void) {
    mylib_shutdown();
    return 1;
}
""", libraries=['mylib'], library_dirs=[os.path.dirname(__file__)])

if __name__ == '__main__':
    ffibuilder.compile(verbose=True)

This works as expected but my test code:

import _mylib_cffi
...

bombs because libmylib.so is not found (it is in current directory, exactly where generated _mylib_cffi.cpython-310-x86_64-linux-gnu.so is located).

I can make it work either moving libmylib.so to /usr/lib (or other directory in the system lib search path) or adding current directory to LD_LIBRARY_PATH.

Both solutions are not particularly appealing to me because I don't want to pollute system settings with needs of a single program.

Is there some way to dynamically load needed lib from within Python?

Note: I'm trying to learn Python-CFFI, so any advice is welcome.

ZioByte
  • 2,690
  • 1
  • 32
  • 68

2 Answers2

0

Following this answer I was able to make it work using:

#!/usr/bin/env python3
import os
import sys
if 'LD_LIBRARY_PATH' not in os.environ:
    os.environ['LD_LIBRARY_PATH'] = os.path.dirname(os.path.realpath(__file__))
    try:
        # print('restarting...')
        os.execve(sys.argv[0], sys.argv, os.environ)
    except Exception as exc:
        print('Failed re-exec:', exc)
        sys.exit(1)
# else:
#     print(f'LD_LIBRARY_PATH={os.environ["LD_LIBRARY_PATH"]}')

import struct
import time
from argparse import ArgumentParser
from xml.etree import ElementTree

import _mylib_cffi
...

I will not accept my own answer though, hoping for a less convoluted strategy.

One evident drawback is this won't work under Windows (yes, I have a libmylib.dll equivalent to libmylib.so) and it would be beneficial to have a multi-platform solution.

ZioByte
  • 2,690
  • 1
  • 32
  • 68
0

At least with GCC on Linux, you can avoid the LD_LIBRARY_PATH hack (described in your own answer to the question) with the linker's -rpath option. I think what you need is:

ffibuilder.set_source(..., extra_link_args=['-Wl,-rpath=$ORIGIN'])

I didn't test it, but I think this should make CFFI produce the glue .so in such a way that this .so should look for libmylib.so in the same directory as where it is.

On Windows, I think nothing is needed: a DLL always (by default) tries to look for other DLLs it depends on in the same directory.

Armin Rigo
  • 12,048
  • 37
  • 48