It's totally possible to make SWIG and ctypes work together, in various different ways, as I'll show with a few examples below. However with that said there is a big caveat: in my experience it's almost never going to get you something that's maintainable, portable, extensible, etc. so I've tended to favour alternatives options where possible. (Those probably include asking the original authors for the source using whatever means of persuasion is appropriate, re-writing the whole thing from scratch, or simply using an alternative library).
Anyway, let's assume you've got a SWIG wrapped library built for a given Python version. And for whatever reason they didn't wrap your favourite function. So you'd like to just use ctypes to monkey patch something into the SWIG generated code. That can actually be pretty easy, provided:
- The functions you want to call only take and return objects by pointer, not by value.
- The functions you want to call only return existing instances of your types and don't malloc new ones. (There's a workaround for that, but it starts to make stuff difficulty again, really fast).
- The structs you care about actually wrapped all the members you cared about and (if it's C++ code) they're POD types.
Your case doesn't really match those constraints, but for a warm-up exercise let's take a look at doing this with the following "easy_mode" function because it does let us introduce one key point:
struct point *easy_mode(struct point *a, struct point *b) {
a->x += b->x;
a->y += b->y;
return a;
}
This function is pretty easy to use given an existing SWIG wrapper that doesn't already wrap it, but does wrap the struct. We can simply use the SWIG code to create instances of the struct and the extract a pointer from them to give (via ctypes) to the exported but not wrapped function, for example:
from ctypes import *
from test import point
p1 = point()
p2 = point()
p1.x = 123
p1.y = 156
p2.x = 123
p2.y = 156
so = CDLL('./_test.so')
so.easy_mode(int(p1.this), int(p2.this))
print(p1.x)
print(p1.y)
Is sufficient to call this function and as we know the return type is really just modifying p1 we can use that knowledge and roll with it. The key thing to take away from this though is that calling int(p1.this) gets you an integer representation of the pointer that the SWIG object is proxying for. And that's all ctypes needs for a pointer.
Let's take this forward though, to the case where we pass and return structs by value. This is much harder because the way the function gets called depends on the size and members of the struct. Their types and ordering matters. And it varies from architecture to architecture. It can even vary within a given architecture based on various things. Luckily ctypes (via libffi, which is an interesting thing in and of itself if you've never seen it before) hides all that from us.
So now our targeted missing function can be something like this:
struct point add_them(struct point a, struct point b) {
struct point ret = { a.x + b.x, a.y + b.y };
return ret;
}
The problem is that in the scenario of having just an existing SWIG module that doesn't call it we know absolutely nothing about the members of struct point. That's critical to being able to call by value. Sure we can make some guesses and if it's simple enough to just guess then you may as well do that and be done with it for ctypes.
Fortunately the existence of a useable SWIG wrapping of the struct gives us (if a few assumptions hold true) enough to make at least a good enough guess as to the types/layout of the struct. Furthermore since we know how to get a pointer to the underlying memory an instance of the struct uses we can construct tests that will show us things about the layout. If all goes well we can use that to construct a ctypes Structure definition of fields that will be compatible.
The crux of this is that we're going to memset an instance to 0 and then try to set each byte to a marker value one at a type, using the SWIG generated setter code for each member. When we inspect that we can make the deductions we need.
Before we can do that however it's helpful to have an upperbound on the size of the struct. We can get that by calling malloc_useable_size() which tells us what the heap allocation got rounded up to. So we can do something like this:
useable_size = CDLL(None).malloc_usable_size
upper_size_bound = useable_size(int(p1.this))
buffer_type = POINTER(c_char * upper_size_bound)
print('Upper size bound is %d' % upper_size_bound)
choose_type = dict([(1, c_uint8), (2, c_uint16), (4, c_uint32)]).get
def generate_members(obj):
for member_name in (x for x in dir(obj) if not x[0] == '_' and x != 'this'):
print('Looking at member: %s' % member_name)
def pos(shift):
test = point()
memset(int(test.this), 0, upper_size_bound)
pattern = 0xff << (8 * shift)
try:
setattr(test, member_name, pattern)
except:
return -1
return bytes(cast(int(test.this), buffer_type).contents).find(b'\xff')
a=[pos(x) for x in range(upper_size_bound)]
offset = min((x for x in a if x >= 0))
size = 1 + max(a) - offset
print('%s is a %d byte type at offset %d' % (member_name, size, offset))
pad = [('pad', c_ubyte * offset)] if (offset > 0) else []
class Member(Structure):
_pack_ = 1
_fields_ = pad + [(member_name, choose_type(size))]
yield Member
This takes every member that SWIG knows about for the given struct and, having computed an array working out the offset of each byte of a given field then computes both the offset and size of that member. (The use of min/max means it should work with both BE and LE hardware). We can take the size and map it onto a type. Given the knowledge we now have we could compute the layout that best matches what we learned. I cheated though and for each member generated a struct that has padding added at the beginning to position the member at exactly the offset we computed. The python code above is a generator that yields a ctypes Structure that has a member of the right size/type at the right offset.
In reality you'll need to deduce a lot more. Floating point numbers would probably be best served with known exactly representable values. We need to consider both signed and unsigned types. Arrays, strings and even nested types could be done. It's all possible on a trial+error basis but that's left as an exercise for the reader.
Finally we need to pull it together. Since I cheated with the one structure per member thing above all we need to do is merge these into a union:
class YourType(Union):
_pack_ = 1
_fields_ = list(zip(string.ascii_lowercase, generate_members()))
_anonymous_ = string.ascii_lowercase[:len(_fields_)]
With that we've now got enough to work with to call our add_them function:
MyType=YourType
y=MyType()
y.x = 1
y.y = 2
add_them = so.add_them
add_them.argtypes = [MyType, MyType]
add_them.restype = MyType
v=add_them(y,y)
print(v)
print('V: %d,%d' % (v.x, v.y))
Which does actually work to call a ctypes function using just information derived from a pre-existing SWIG module.
I'd still recommend not doing it in any real code though!
