9

I was wondering that, if I opened my own dll library compiled from custom c code, like this:

import ctypes
my_lib = ctypes.cdll.LoadLibrary('./my_dll.dll')
my_func = my_lib.my_func
# Stuff I want to do with func()

Do I need to close the my_lib object after use, like a file object? Will doing this make the code cleaner, more efficient, and more "pythonic"?

Thanks!

Yuxiang Wang
  • 8,095
  • 13
  • 64
  • 95

1 Answers1

14

Generally you shouldn't have to free a shared library. Consider that CPython doesn't provide a means to unload a regular extension module from memory. For example, importing sqlite3 will load the _sqlite3 extension and sqlite3 shared library for the life of the process. Unloading extensions is incompatible with the way CPython uses pointers as object IDs. Accessing a deallocated (and possibly reused) address would be undefined behavior.

If you need to unload or reload a shared library, and are confident that it's safe, the _ctypes extension module has POSIX dlclose and Windows FreeLibrary, which call the system functions of the same name. Both take the library handle as the single argument. This is the _handle attribute of a CDLL instance. If unloading the library fails, OSError is raised.

Both dlclose and FreeLibrary work by decrementing the handle's reference count. The library is unloaded when the count is decremented to 0. The count is initially 1 and gets incremented each time POSIX dlopen or Windows LoadLibrary is called for an already loaded library.

POSIX Example

#include <stdio.h>

void __attribute__((constructor)) initialize()
{
    printf("initialize\n");
}

void __attribute__((destructor)) finalize()
{
    printf("finalize\n");
}

POSIX Python

>>> import ctypes
>>> lib1 = ctypes.CDLL('./lib.so')
initialize
>>> lib2 = ctypes.CDLL('./lib.so')
>>> lib1._handle == lib2._handle
True

>>> import _ctypes
>>> _ctypes.dlclose(lib1._handle)
>>> _ctypes.dlclose(lib1._handle)
finalize

>>> lib1 = ctypes.CDLL('./lib.so')
initialize
>>> _ctypes.dlclose(lib1._handle)
finalize

Windows Example

#include <stdio.h>
#include <windows.h>

void initialize()
{
    printf("initialize\n");
}

void finalize()
{
    printf("finalize\n");
}

BOOL WINAPI DllMain(HINSTANCE hinstDLL,
                    DWORD fdwReason,
                    LPVOID lpReserved)
{
    switch(fdwReason) 
    { 
        case DLL_PROCESS_ATTACH:
            initialize();
            break;

        case DLL_PROCESS_DETACH:
            finalize();
    }
    return TRUE;
}

Windows Python

>>> import ctypes
>>> lib1 = ctypes.CDLL('./lib.dll')
initialize
>>> lib2 = ctypes.CDLL('./lib.dll')
>>> lib1._handle == lib2._handle
True

>>> import _ctypes
>>> _ctypes.FreeLibrary(lib1._handle)
>>> _ctypes.FreeLibrary(lib1._handle)
finalize

>>> lib1 = ctypes.CDLL('./lib.dll')  
initialize
>>> _ctypes.FreeLibrary(lib1._handle)
finalize
Eryk Sun
  • 33,190
  • 5
  • 92
  • 111
  • on win7epX64 with py2.7x32 I can load my specific dll 2 times but three or more tries result in `WindowsError: [Error -2147483645] One or more arguments are invalid` on `self._handle = _dlopen(self._name, mode)` at `ctypes\__init__.py", line 365, in __init__` and python.exe crashing at exit. **Not** the case with a standard dll like `msvcrt40.dll` but still unclear what on Earth possibly can go wrong with my other dll. – n611x007 May 06 '14 at 11:52
  • 1
    I doubt the formatted error message is relevant. Error code `0x80000003` is most likely an NT `STATUS_BREAKPOINT`. Maybe it called [`DebugBreak`](http://msdn.microsoft.com/en-us/library/windows/desktop/ms679297%28v=vs.85%29.aspx). You'll need a debugger such as windbg or cdb to get a stack trace. Try to reproduce the error outside of Python in a C console program that calls `LoadLibrary`. – Eryk Sun May 06 '14 at 20:23
  • 1
    There are definitely times you need to unload a DLL. This is one of the less well thought-out bits of Python--it was designed by Unix people who apparently didn't know that you can't replace a DLL in Windows if it's loaded like you can on Unix, and you need to be able to unload in plugin environments in order to upgrade native code without restarting the whole application. – Glenn Maynard May 31 '16 at 05:36
  • @GlennMaynard, you can replace a loaded DLL (on disk). Just rename the loaded DLL to some name or directory that's on the same volume. You can do this because the file object for the DLL is opened with delete sharing. OTOH, replacing the mapping in a process requires unloading the DLL and reloading it. I don't know if you're guaranteed to get the same base address, which would really a show stopper. – Eryk Sun May 31 '16 at 05:44
  • @GlennMaynard, to clarify delete sharing of DLLs, the sharing mode itself allows deleting and renaming a DLL, but you can't actually delete the file because it's mapped by the memory manager, which applies equally to data file mappings and paging files. Note that the delete doesn't fail with a sharing violation like you would normally get from trying to delete an open file that wasn't opened with delete sharing. – Eryk Sun May 31 '16 at 06:03