forcing ctypes.cdll.LoadLibrary() to reload library from file

Question

I have the following code

import ctypes
lib1 = ctypes.cdll.LoadLibrary("./mylib.so")
# modify mylib.so (code generation and compilation) or even delete it
lib2 = ctypes.cdll.LoadLibrary("./mylib.so")

The problem is that lib2 refers to the original shared library, not the new one. If I delete mylib.so between the calls I get no error.

Using ctypes._reset_cache() does not help.

How can I tell ctypes to actually reload the library from the hard disk?

Does this answer your question? [How can I unload a DLL using ctypes in Python?](https://stackoverflow.com/questions/359498/how-can-i-unload-a-dll-using-ctypes-in-python) — Markus Dutschke, Oct 22 '20 at 13:34

CristiFati · Accepted Answer · 2023-01-11T10:48:16.787

I don't know how to instruct ctypes how to unload a library (didn't find a way on [Python.Docs]: ctypes - A foreign function library for Python, but that doesn't mean that there isn't one).

It can be done manually, by forcing the loader to (decrement the library's reference count and) unload it via [Man7]: DLCLOSE(3P) (also read [Man7]: DLOPEN(3) for additional info on loading / unloading libraries).

dll00.c:

#include <stdio.h>


int func0(int arg0)
{
    int alter_factor = 2;
    printf("From C - arg0: %d, alter_factor: %d\n", arg0, alter_factor);
    return arg0 * alter_factor;
}

code00.py:

#!/usr/bin/env python

import ctypes as cts
import sys


DLL_NAME = "./dll00.{:s}".format("dll" if sys.platform[:3].lower() == "win" else "so")


def handle_dll(name=DLL_NAME):
    dll = cts.CDLL(name)
    func0 = dll.func0
    func0.argtypes = (cts.c_int,)
    func0.restype = cts.c_int
    return dll, func0


def main(*argv):
    dlclose_func = cts.CDLL(None).dlclose
    dlclose_func.argtypes = (cts.c_void_p,)
    dlclose_func.restype = cts.c_int

    dll, func0 = handle_dll()
    res = func0(42)
    print(res)
    dlclose_func(dll._handle)
    input("In another terminal, modify the C code (e.g. change `alter_factor`), recompile (gcc -fPIC -shared -o dll00.so dll00.c), and when done press ENTER here... ")
    dll, func0 = handle_dll()
    res = func0(42)
    print(res)


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

Output:

(qaic-env) [cfati@cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q050964033]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[064bit prompt]> ls
code00.py  dll00.c  dll00.so
[064bit prompt]>
[064bit prompt]> python code00.py
Python 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0] 064bit on linux

From C - arg0: 42, alter_factor: 2
84
In another terminal, modify the C code (e.g. change `alter_factor`), recompile (gcc -fPIC -shared -o dll00.so dll00.c), and when done press ENTER here... 
From C - arg0: 42, alter_factor: 3
126

Done.

Related (more or less):

That worked. Thanks. However, this method might provoke segmentation faults (which seems to be a matter of fact). Also, it should be noted, that one has to call dlclose_func multiple times if the lib was loaded multiple times. — cknoll, Jun 24 '18 at 17:09
Yes, trying to access a symbol from an unloaded library is *Undefined Behavior* (and expected to segfault sometimes). And also each `dlopen` call must have its `dlaclose` counterpart. But that's all knowledge related to loading/unloading libs. — CristiFati, Jun 24 '18 at 17:35

score 1 · Answer 2 · answered May 26 '20 at 11:50

Simplifying CristiFatis answer a bit, I wrote a close-library function. The following code can be used to develop in your shared library and call it (in the most recent version) from python.

import ctypes
def ctypesCloseLibrary(lib):
    dlclose_func = ctypes.CDLL(None).dlclose
    dlclose_func.argtypes = [ctypes.c_void_p]
    dlclose_func.restype = ctypes.c_int

    dlclose_func(lib._handle)

if __name__== "__main__":
    lib = ctypes.cdll.LoadLibrary('./mylib.so')

    # do things with lib_MyClass

    ctypesCloseLibrary(lib)

Just call ctypesCloseLibrary when you want lib to be reloadable by lib = ctypes.cdll.LoadLibrary('./mylib.so').

forcing ctypes.cdll.LoadLibrary() to reload library from file

2 Answers2

Linked