What is the best way of supporting 128-bit integers (currently __uint128_t) with Python ctypes?
A user-defined struct of two uint64_t's perhaps, but this will create alignment issues where that is required.
Any thoughts on why ctypes has not been extended to support 128-bit integers?
If you really want to work with 128-bit integers then you don't need to worry about alignment. No current architecture, no machine that Python runs on, supports 128-bit native integer arithmetic. So no machine would require or benefit from having 128-bit integers 16-byte aligned. Just use that user defined struct and you'll be fine.
If what you're really asking for is support for 128-bit vector types then you'll probably need to have them aligned. That is, you need them aligned if you're creating them in Python code and passing them by reference to C/C++ code. You can't reliably pass them by value there's no way to get ctypes to align them properly on the stack (if that's required by the architecture ABI). Vectors that are passed from C/C++ to Python will presumably already be aligned properly. So, if you can arrange it so all your vectors are allocated in C/C++ code you should also be fine with your user defined struct.
Assuming that you do really need to create aligned vectors in Python code then I've included code for aligned ctypes arrays. I also have code to align other ctypes types that I haven't included to the code size reasonable. Arrays should be enough for most purposes. These aligned arrays have a couple of limitations. They won't be aligned properly if you pass them by value to C/C++ functions or if you include them as members in a struct or union. You can make aligned arrays of aligned arrays using the * operator.
Use aligned_array_type(ctypes-type, length, alignment) to create new aligned array types. Use aligned_type(ctypes-type, alignment) to create an aligned version of an already existing array type.
import ctypes
ArrayType = type(ctypes.Array)
class _aligned_array_type(ArrayType):
def __mul__(self, length):
return aligned_array_type(self._type_ * self._length_,
length, self._alignment_)
def __init__(self, name, bases, d):
self._alignment_ = max(getattr(self, "_alignment_", 1),
ctypes.alignment(self))
def _aligned__new__(cls):
a = cls._baseclass_.__new__(cls)
align = cls._alignment_
if ctypes.addressof(a) % align == 0:
return a
cls._baseclass_.__init__(a) # dunno if necessary
ctypes.resize(a, ctypes.sizeof(a) + align - 1)
addr = ctypes.addressof(a)
aligned = (addr + align - 1) // align * align
return cls.from_buffer(a, aligned - addr)
class aligned_base(object):
@classmethod
def from_address(cls, addr):
if addr % cls._alignment_ != 0:
raise ValueError, ("address must be %d byte aligned"
% cls._alignment_)
return cls._baseclass_.from_address(cls, addr)
@classmethod
def from_param(cls, addr):
raise ValueError, ("%s objects may not be passed by value"
% cls.__name__)
class aligned_array(ctypes.Array, aligned_base):
_baseclass_ = ctypes.Array
_type_ = ctypes.c_byte
_length_ = 1
__new__ = _aligned__new__
_aligned_type_cache = {}
def aligned_array_type(typ, length, alignment = None):
"""Create a ctypes array type with an alignment greater than natural"""
natural = ctypes.alignment(typ)
if alignment == None:
alignment = typ._alignment_
else:
alignment = max(alignment, getattr(typ, "_alignment_", 1))
if natural % alignment == 0:
return typ * length
eltsize = ctypes.sizeof(typ)
eltalign = getattr(typ, "_alignment_", 1)
if eltsize % eltalign != 0:
raise TypeError("type %s can't have element alignment %d"
" in an array" % (typ.__name__, alignment))
key = (_aligned_array_type, (typ, length), alignment)
ret = _aligned_type_cache.get(key)
if ret == None:
name = "%s_array_%d_aligned_%d" % (typ.__name__, length,
alignment)
d = {"_type_": typ,
"_length_": length,
"_alignment_": alignment}
ret = _aligned_array_type(name, (aligned_array,), d)
_aligned_type_cache[key] = ret
return ret
def aligned_type(typ, alignment):
"""Create a ctypes type with an alignment greater than natural"""
if ctypes.alignment(typ) % alignment == 0:
return typ
if issubclass(typ, ctypes.Array):
return aligned_array_type(typ._type_, typ._length_,
alignment)
else:
raise TypeError("unsupported type %s" % typ)
