Python dict keyed by class has KeyErrors

Question

I am writing a program to process serial traffic using ctypes. At the same time, I'm interfacing with a database library which uses different-but-similar classes to apply fields to buffers of data.

I wanted to write a function which could take an arbitrary ctypes Structure, iterate over its fields, and make a call into the database library. To do this, I made a map of {ctypes class : database class} and was getting bizarre KeyErrors. However it turns out the database library has nothing to do with it and you can see the same behavior with a dict of {ctypes class : string}, as in the below minimal example:

from ctypes import *

db_map = {
    c_char : "DByte",
    c_byte : "DByte",
    c_ubyte : "DByte",
    c_ushort : "DUShort",
    c_uint16 : "DUShort",
}

class command_buff(BigEndianStructure):
    _pack_   = 1
    _fields_ = [("frame_header",    c_char   ),
                ("command_id",      c_uint8  ), 
                ("param_value",     c_uint8  ),
                ("crc16",           c_uint16 ),
                ("frame_footer",    c_char   )]

def display(buff, database_name):
    """Applies my structure to the Dbuffer named database_name."""
    global db_map
    for key in db_map:
        print(f"{key} {id(key)}")
    print()
    for field_name, c_typ, *rest in buff._fields_:
        stol_typ = db_map.get(c_typ, None)
        if stol_typ is None:
            print(f"  ERROR Can't find type {c_typ} for name {field_name}")
            print(f"  ERROR ({field_name}, {id(c_typ)}, {rest})")
        else:
            print(f"{database_name}.{field_name}")

cb = command_buff
display(cb, "Foo")

Try it online!

Which produces:

<class 'ctypes.c_char'> 2337600989576
<class 'ctypes.c_byte'> 2337600987688
<class 'ctypes.c_ubyte'> 2337600959368
<class 'ctypes.c_ushort'> 2337600969752

Foo.frame_header
Foo.command_id
Foo.param_value
  ERROR Can't find type <class 'ctypes.c_ushort'> for name crc16
  ERROR (crc16, 2337600963144, [])
Foo.frame_footer

As you can see, the class 'ctypes.c_ushort' in the dict has a different ID than that class 'ctypes.c_ushort' in the _fields_ member, which is presumably why it thinks it isn't in the dict. But I don't understand how that could be the case, considering both of them came from the exact same import statement.

I have looked at questions such as this one, but most seem to deal with multiple instances of a class having different IDs. Here, it seems the class itself has multiple IDs, even over such a short program.

What is the behind-the-scenes explanation for why is this happening?

What is the correct way to key a dict by class, or (if that's a silly thing to do) to achieve the goal of mapping class -> class?

Answer 1

This is an internal quirk of how ctypes' BigEndianStructure works.

Digging a little into ctypes/_endian.py, I found out that the ctypes types have internal types for the explicit-endian versions of themselves (__ctype_be__ and __ctype_le__).

The metaclass for the endianed structure types swap the endianness of types in _fields_ on other-endian machines.

If you have a BigEndianStructure, your mapping also needs to use those BigEndian (_be) types:

db_map = {
    c_char.__ctype_be__: "DByte",
    c_byte.__ctype_be__: "DByte",
    c_ubyte.__ctype_be__: "DByte",
    c_ushort.__ctype_be__: "DUShort",
    c_uint16.__ctype_be__: "DUShort",
}

The other option, with less underscores, might be to just use the __name__ of the type and have a stringly typed dict:

db_map = {
    "c_char": "DByte",
    "c_byte": "DByte",
    "c_ubyte": "DByte",
    "c_ushort_be": "DUShort",
    "c_ushort_le": "DUShort",
    "c_uint16": "DUShort",
}

# ...

db_map.get(c_typ.__name__)