python ctypes garbage collection

Question

I am using ctypes to access a c library (it is really c++ code) from python:

class CRMeAnnotationList(object):
def __init__(self):
    self.obj = lib.CRMeAnnotationList_new()de
def __del__(self):
    if self.obj:
        lib.CRMeAnnotationList_destroy(self.obj)
lst = CRMeAnnotationList()

The C code looks like this

extern "C" {__declspec(dllexport) CRMeAnnotationList* CRMeAnnotationList_new() 
                { return new CRMeAnnotationList(); 
                }
            __declspec(dllexport)void CRMeAnnotationList_destroy(CRMeAnnotationList* p)
                {
                if (p)
                    delete p;
                }
            }

This code gives me finished with exit code -1073741819 (0xC0000005)

whereas if I do not destroy the pointer I get 0 as exit code.

Does this mean, that I do not need to destroy that pointer i.e execute the destructor part del in the python code?

You should not use del. it’s not working the way you think it works. And can cause weird problems, posisbly the one you see. Try allocating your C++-object without a wrapper class, and then deallocate. And see what happens. And there is no magic removing the Object, so you DO need to call the destroy. — deets, Jan 06 '19 at 21:34
Show a complete [mcve]. Have you set `.argtypes` and `.restype` for your functions? Did you create `lib` correctly? At a guess, you are using 64-bit Python so and the pointer returned by `new` is truncated to 32-bits due to not setting `.restype`. The default return type is `c_int` (32-bits). — Mark Tolonen, Jan 06 '19 at 21:38
@deets: Can you justify that position? `__del__` looks very suitable to me here. — Eric, Jan 06 '19 at 22:03
The looks are deceiving ;) There is actually no guarantee that `__del__` is being run at all. If it is run, there is no guarantee about the order of things - a common occurrence is that necessary modules (e.g. ctypes in this case) are being GCed *before* the destructor is run, obviously making it fail. This is one of the reasons `with` has been introduced. For a discussion with follow-ups, see https://stackoverflow.com/questions/6104535/i-dont-understand-this-python-del-behaviour — deets, Jan 07 '19 at 16:12

score 1 · Answer 1 · answered Jan 06 '19 at 22:01

1

You are likely running 64-bit Python and haven't set .argtypes and .restype properly.

Example:

test.cpp

#define API __declspec(dllexport) // Windows-specific

extern "C" {

struct CRMeAnnotationList {
    int a;
};

API CRMeAnnotationList* CRMeAnnotationList_new() {
    return new CRMeAnnotationList();
}

API void CRMeAnnotationList_destroy(CRMeAnnotationList* p) {
    if(p)
        delete p;
}

}

test.py

from ctypes import *

lib = CDLL('test')
lib.CRMeAnnotationList_new.argtypes = None
lib.CRMeAnnotationList_new.restype = c_void_p
lib.CRMeAnnotationList_destroy.argtypes = c_void_p,
lib.CRMeAnnotationList_destroy.restype = None

class CRMeAnnotationList:

    def __init__(self):
        print(f'created   {id(self):#x}')
        self.obj = lib.CRMeAnnotationList_new()

    def __del__(self):
        print(f'destroyed {id(self):#x}')
        if self.obj:
            lib.CRMeAnnotationList_destroy(self.obj)

lst = CRMeAnnotationList()

Output:

created   0x2232dfc35c0
destroyed 0x2232dfc35c0

Output when .argtypes and .restype lines are commented out:

created   0x1c98e8f35c0
destroyed 0x1c98e8f35c0
Exception ignored in: <function CRMeAnnotationList.__del__ at 0x000001C98E9176A8>
Traceback (most recent call last):
  File "C:\Users\metolone\Desktop\test.py", line 16, in __del__
OSError: exception: access violation reading 0xFFFFFFFF8C6CABCF

Note the address of the access violation. It is a 32-bit value sign-extended to 64-bits.

answered Jan 06 '19 at 22:01

Mark Tolonen

166,664
26
169
251

Should `argtypes` be set to `()` instead of `None` for the function taking 0 arguments? – Eric Jan 06 '19 at 22:05
@Eric Note `tuple()` not `()` is an empty tuple. The docs do state passing tuple of arguments, but 'None` and `[]` work correctly as well. – Mark Tolonen Jan 06 '19 at 22:19
@Mark('32bit', 'WindowsPE') – prudnik Jan 06 '19 at 22:41
@prodnik Are you saying you are running 32-bit WindowsPE? Set `.argtypes` and `.restype` anyway, but update your question with a [mcve] that reproduces the problem, as I have, if you want more help. – Mark Tolonen Jan 06 '19 at 22:44
@Mark print (platform.architecture()) gives ('32bit', 'WindowsPE') and everything else works fine. (The code gets some complex data from an xml file.) But thanks for the example, I have a closer look. And by the way i am planning to use 64 bit python 3, but for now this is pythonn 2.7 code. – prudnik Jan 06 '19 at 22:48
@prudnik My systems says `('64-bit','windowsPE')` but it is Windows 10. `platform.platform()` seems to be more accurate. In any case update your question with an an [mcve] and mention OS and Python version. – Mark Tolonen Jan 06 '19 at 22:58
_"`tuple()` not `()` is an empty tuple"_ - Not only are both empty tuples, but both are the _same_ empty tuple. `tuple() is ()` gives `True` on my machine. – Eric Jan 06 '19 at 23:05
I have set .argtypes and .restype like you said. I exits with code 0. So that seems to be the solution to the problem. Thanks a lot. – prudnik Jan 06 '19 at 23:07
@Eric Sorry, I was thinking of for example `(5)` vs. `(5,)`. The first is not a tuple. I'm in the habit of not relying on parentheses for tuples, since `x = 5,` and `x=1,2,3` are also a tuples. *Explicit is better than implicit* :^) – Mark Tolonen Jan 06 '19 at 23:35

python ctypes garbage collection

1 Answers1