Equivalent C structure in Python

Question

I have the following in C:

typedef struct {
    short          Whole;
    unsigned short Frac;
} FirstStruct, FAR *pFirstStruct;

typedef struct {
    char FirstArr[3];
    FirstStruct SecondArr[3][3]
} SecStruct, FAR * pSecStruct;

I would like to do something similar in Python. Found this answer explaining how to use ctypes for this purpose, but I am having problems with SecondArr[3][3]. Here's the code in Python:

class FirstStruct(ctypes.Structure):
     _pack_ = 2
     _fields = [("Whole", ctypes.c_short),
                ("Fract", ctypes.c_ushort)]
 
 
class SecStruct(ctypes.Structure):
    class _A(ctypes.Array):
        _type_ = ctypes.c_char
        _length_ = 3

    class _B(ctypes.Array):
        _type_ = FirstStruct
        _length_ = 3

    class _C(ctypes.Array):
        _type_ = _B
        _length_ = 3

    _pack_ = 2
    _fields = [("FirstArr", _A),
               ("SecondArr", _C)]

By doing that, Pylance complains that "_B" is not defined, and I'm not completely sure it will work, nor if it is safe to mix two subclasses in that way to create a new C structure.

Is this the correct way of doing it even if Pylance complains about it, or is there any other way to convert the structure mentioned?

"C" is not C++ (those are already two different languages) and concepts from one language do not always translate (well) to another language. So try to find something that works well in python that does WHAT you want it to do, not how "C" does it. — Pepijn Kramer, Feb 05 '23 at 10:31
@PepijnKramer I have changed the tag as suggested. I am trying to achieve that because the program has to talk with devices that connect to a computer and follow a specification that is witten in C. Therefore, to manage those devices from a Python app, I have to create to "translate" the instructions from C to Python, using ctypes. — Jose Vega, Feb 05 '23 at 10:33
Are you working with some form of binary data? That is, are you trying to exactly match the C structure because you're reading structured data from a file (or other source) into it? Without understanding your use case it's hard to give you the best answer. — larsks, Feb 05 '23 at 11:32
Just move the definition of _B (and _A and _C while you are at it) out of SecStruct. There is no real reason to make them nested. — n. m. could be an AI, Feb 05 '23 at 12:24
@larsks exactly. I'm trying to match a C structures that comes from a file. The code in Python must match that structure to be able to convert it into an in-memory file, manipulate it, and then save it with certain properties. — Jose Vega, Mar 06 '23 at 00:07

CristiFati · Answer 1 · 2023-02-06T09:19:00.217

According to [Python.Docs]: ctypes - Arrays (emphasis is mine):

Arrays are sequences, containing a fixed number of instances of the same type.

The recommended way to create array types is by multiplying a data type with a positive integer:
TenPointsArrayType = POINT * 10

Also (not sure whether it's a typo) for structures, the attribute name holding member data is _fields_ (also ends with an UnderScore and not _fields (as in your case)).
I'm not going to discuss the problems (NameErrors) in your code, as they are generated by a misunderstanding and (as a consequence) are a totally different matter.

Here's a small example. Note that I declared the array types gradually (for readability), but they can be also declared on the fly (only where they are needed).
Instances can then be manipulated like in C.

code00.py:

#!/usr/bin/env python

import ctypes as cts
import sys


class FirstStruct(cts.Structure):
    _pack_ = 2
    _fields_ = (
        ("Whole", cts.c_short),
        ("Fract", cts.c_ushort),
    )

FirstStructPtr = cts.POINTER(FirstStruct)


CharArr3 = cts.c_char * 3
FirstStructArr3 = FirstStruct * 3
FirstStructArr3Arr3 = FirstStructArr3 * 3  # FirstStruct * 3 * 3  # Equivalent


class SecStruct(cts.Structure):
    _pack_ = 2
    _fields_ = (
        ("FirstArr", CharArr3),
        ("SecondArr", FirstStructArr3Arr3),
    )

SecStructPtr = cts.POINTER(SecStruct)


def main(*argv):
    ss = SecStruct()
    print("FirstArr:", ss.FirstArr)
    ss.FirstArr = b"XYZ"
    print("FirstArr:", ss.FirstArr, ss.FirstArr[2])
    print("SecondArr:", ss.SecondArr)
    print("SecondArr[0]:", ss.SecondArr[0])
    print("SecondArr[0][0]:", ss.SecondArr[0][0])
    print("SecondArr[0][0].Whole", ss.SecondArr[0][0].Whole)
    ss.SecondArr[0][0].Whole = 0xFF
    print("SecondArr[0][0].Whole", ss.SecondArr[0][0].Whole)


if __name__ == "__main__":
    print("Python {:s} {:03d}bit on {:s}\n".format(" ".join(elem.strip() for elem in sys.version.split("\n")),
                                                   64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    rc = main(*sys.argv[1:])
    print("\nDone.\n")
    sys.exit(rc)

Output:

[cfati@CFATI-5510-0:e:\Work\Dev\StackOverflow\q075351547]> "e:\Work\Dev\VEnvs\py_pc064_03.10_test0\Scripts\python.exe" ./code00.py
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] 064bit on win32

FirstArr: b''
FirstArr: b'XYZ' 90
SecondArr: <__main__.FirstStruct_Array_3_Array_3 object at 0x000001C128675F40>
SecondArr[0]: <__main__.FirstStruct_Array_3 object at 0x000001C128BDC9C0>
SecondArr[0][0]: <__main__.FirstStruct object at 0x000001C128BDC2C0>
SecondArr[0][0].Whole 0
SecondArr[0][0].Whole 255

Done.

Might also worth reading:

[SO]: C function called from Python via ctypes returns incorrect value (@CristiFati's answer) for a common pitfall when working with CTypes (calling functions)
[SO]: C++ & Python: Pass and return a 2D double pointer array from python to c++ (@CristiFati's answer) for an example of creating multidimensional (2D in that case) arrays. Dimension order is important (maybe a bit counterintuitive at the beginning), but dimensions should be specified in reversed order than in C
[SO]: Dynamically defining/updating ctypes structure in Python (@CristiFati's answer)

score 1 · Answer 2 · answered Feb 05 '23 at 20:47

Below shows multiplying a type by an integer creates an array, but note that C stores arrays in row-major order. To create the arrays with the correct memory layout, multiply by the column size first to create a single row array, then by the row size to complete the array. In your case of [3][3] it won't matter, but it will if the sizes are different, as I've intentionally shown.

A __repr__ function is also included with each struct to define how the structure can display itself and an instance is created from a consecutively-numbered byte buffer to illustrate the correct little-endian layout:

import ctypes as ct

ROWS = 3
COLS = 2

class FirstStruct(ct.Structure):
    _fields_ = (("Whole", ct.c_short),
                ("Frac", ct.c_ushort))

    def __repr__(self):
        return f'FirstStruct(Whole={self.Whole:#x}, Frac={self.Frac:#x})'

class SecStruct(ct.Structure):
    _fields_ = (("FirstArr", ct.c_ubyte * COLS),           # equivalent to FirstArr[COLS]
                ("SecondArr", FirstStruct * COLS * ROWS))  # equivalent to FirstStruct[ROWS][COLS]

    def __repr__(self):
        arr = '[' + '\n                     '.join([str(x[:]) for x in self.SecondArr]) + ']'
        return (f'SecStruct(FirstArr={self.FirstArr[:]},\n'
                f"          SecondArr={arr})")
                
s = SecStruct.from_buffer_copy(bytes(range(ct.sizeof(SecStruct))))
print(s)

Output:

SecStruct(FirstArr=[0, 1],
          SecondArr=[[FirstStruct(Whole=0x302, Frac=0x504), FirstStruct(Whole=0x706, Frac=0x908)]
                     [FirstStruct(Whole=0xb0a, Frac=0xd0c), FirstStruct(Whole=0xf0e, Frac=0x1110)]
                     [FirstStruct(Whole=0x1312, Frac=0x1514), FirstStruct(Whole=0x1716, Frac=0x1918)]])

Equivalent C structure in Python

2 Answers2