Why is this Delphi DLL throwing a WindowsError when called from Python?

Question

Environment: Python 2.7.9 (32 bit) on Windows 7 (x64)

I'm using ctypes in Python to access a DLL named "tdbaccess.dll" (found here), which I assume was written in Delphi, based on the author's documentation and this info on the error I'm getting. I don't have access to the source of the DLL, unfortunately. The DLL is used to open a 'database' roster file used by the old Madden NFL PC games (a roster from the 2008 version is what I am using - don't judge :P).

My code contains these two structs:

class tdbTableProperties(Structure):
    _fields_ = [
        ('Name', c_char_p),
        ('FieldCount', c_int),
        ('Capacity', c_int),
        ('RecordCount', c_int),
        ('DeletedCount', c_int),
        ('NextDeletedRecord', c_int),
        ('Flag0', c_bool),
        ('Flag1', c_bool),
        ('Flag2', c_bool),
        ('Flag3', c_bool),
        ('NonAllocated', c_bool),
        ('HasVarchar', c_bool),
        ('HasCompressedVarchar', c_bool),
    ]

and

class tdbFieldProperties(Structure):
    _fields_ = [
        ('Name', c_char_p),
        ('Size', c_int),
        ('FieldType', c_int),
    ]

Instances of these structs get passed by ref to two functions in the DLL which in turn fill out the fields inside. First, the instances of tdbTableProperties are appended to a list and passed as so:

# Create a list to hold all our table properties structs.
listTdbTableProperties = []

# Loop over the tables and get their properties.
for i in range(intNumberOfTables):
    listTdbTableProperties.append(tdbTableProperties(Name="ASDF")) #Initialize Name field per docs
    boolGotTableProperties = tdbaccessDLL.TDBTableGetProperties(intDBIndex, i, byref(listTdbTableProperties[i]))
    if boolGotTableProperties:
        print("listTdbTableProperties[%d].Name = %r" % (i, listTdbTableProperties[i].Name))
        print("listTdbTableProperties[%d].FieldCount = %d\n" % (i, listTdbTableProperties[i].FieldCount))

This works fine and gives the following output:

listTdbTableProperties[0].Name = 'CITY'
listTdbTableProperties[0].FieldCount = 21

listTdbTableProperties[1].Name = 'COCH'
listTdbTableProperties[1].FieldCount = 68

[...]

listTdbTableProperties[5].Name = 'INJY'
listTdbTableProperties[5].FieldCount = 5

listTdbTableProperties[6].Name = 'PLAY'
listTdbTableProperties[6].FieldCount = 110

[...]

I'm really only interested in the "PLAY" table, so now I try a similar thing with instances of tdbFieldProperties, to get the properties for each field in that table...

# A list to hold all our field properties structs for the player table.
listPlayerTableTdbFieldProperties = []

# Get the properties of each of the fields for the PLAY table (index 6 in listTdbTableProperties).
for i in range(listTdbTableProperties[6].FieldCount):
    listPlayerTableTdbFieldProperties.append(tdbFieldProperties(Name="Blah")) #Initialize Name field per docs
    print("listPlayerTableTdbFieldProperties[%d].Name BEFORE = %r" % (i, listPlayerTableTdbFieldProperties[i].Name))
    boolGotTableFieldProperties = tdbaccessDLL.TDBFieldGetProperties(intDBIndex, listTdbTableProperties[6].Name, i, byref(listPlayerTableTdbFieldProperties[i]))
    if boolGotTableFieldProperties:
        print("listPlayerTableTdbFieldProperties[%d].Name AFTER = %r" % (i, listPlayerTableTdbFieldProperties[i].Name))

... However, it eventually fails as so, always at the 70th of the 110 fields in the sixth table:

listPlayerTableTdbFieldProperties[0].Name BEFORE = 'Blah'
listPlayerTableTdbFieldProperties[0].Name AFTER = 'TRV1'
listPlayerTableTdbFieldProperties[1].Name BEFORE = 'TRV1'
listPlayerTableTdbFieldProperties[1].Name AFTER = 'TEZ1'
listPlayerTableTdbFieldProperties[2].Name BEFORE = 'TEZ1'
listPlayerTableTdbFieldProperties[2].Name AFTER = 'TRV2'
[...]
listPlayerTableTdbFieldProperties[68].Name BEFORE = 'TAth'
listPlayerTableTdbFieldProperties[68].Name AFTER = 'TAss'
listPlayerTableTdbFieldProperties[69].Name BEFORE = 'TAss'
Traceback (most recent call last):
  File "DLLtest2.py", line 70, in <module>
    boolGotTableFieldProperties = tdbaccessDLL.TDBFieldGetProperties(intDBIndex, listTdbTableProperties[6].Name, i, byref(listPlayerTableTdbFieldProperties[i]))
WindowsError: [Error 250477278] Windows Error 0xEEDFADE

Accoring to the documentation for tdbaccess.dll, in a Delphi or Visual Basic .NET implementation the tdbFieldProperties struct would have FieldType as an enum, which has no exact representation in Python. From the documentation:

Delphi Syntax  
  type
    TtdbFieldProperties = packed record
      Name: PWideChar;
      Size: Integer;
      FieldType: TtdbFieldType;
  end;

Visual Basic .NET Syntax
  <StructLayout(LayoutKind.Sequential, CharSet:=CharSet.Unicode)>
  Structure TdbFieldProperties
    Public Name As String
    Public Size As Integer
    Public FieldType As TdbFieldType
  End Structure

and

Delphi Syntax
  type
    TtdbFieldType = (tdbString = 0, tdbBinary = 1, tdbSInt = 2, tdbUInt = 3, tdbFloat = 4, tdbVarchar = $D, tdbLongVarchar = $E, tdbInt = $2CE);

Visual Basic .NET Syntax
  Enum TdbFieldType
    tdbString = 0
    tdbBinary = 1
    tdbSInt = 2
    tdbUInt = 3
    tdbFloat = 4
    tdbVarchar = &HD
    tdbLongVarchar = &HE
    tdbInt = &H2CE
  End Enum

So, based on answers to this question, I've tried using c_int, c_uint (and virtually every other ctype) in Python as the type for FieldType. But no matter what I use, I get Windows Error 0xEEDFADE at the 70th iteration of the loop for the 6th table. And not only that, but the values I'm getting for the Name field in the tdbFieldProperties struct are not what they should be even before the error. If I expand my code to get the field properties for each field in each table, then every table I query puts the same values in the Name field in the same order, so clearly something is wrong even before the error itself:

# Loop over the fields in each tdbtpTableProperties and get the properties of each field.
tdbfpTableFieldProperties = [[] for x in range(intNumberOfTables)]

for i in range(intNumberOfTables):
    print("\n")
    for j in range(tdbtpTableProperties[i].FieldCount):
        tdbfpTableFieldProperties[i].append(tdbFieldProperties(Name="Blah"))
        print("tdbfpTableFieldProperties[%d][%d].Name BEFORE = %r" % (i, j, tdbfpTableFieldProperties[i][j].Name))
        boolGotTableFieldProperties = tdbaccessDLL.TDBFieldGetProperties(intDBIndex, tdbtpTableProperties[i].Name, j, byref(tdbfpTableFieldProperties[i][j]))
        print("tdbfpTableFieldProperties[%d][%d].Name AFTER = %r" % (i, j, tdbfpTableFieldProperties[i][j].Name))

Which gives:

tdbfpTableFieldProperties[0][0].Name BEFORE = 'Blah'
tdbfpTableFieldProperties[0][0].Name AFTER = 'TRV1'
tdbfpTableFieldProperties[0][1].Name BEFORE = 'TRV1'
tdbfpTableFieldProperties[0][1].Name AFTER = 'TEZ1'
tdbfpTableFieldProperties[0][2].Name BEFORE = 'TEZ1'
tdbfpTableFieldProperties[0][2].Name AFTER = 'TRV2'
tdbfpTableFieldProperties[0][3].Name BEFORE = 'TRV2'
tdbfpTableFieldProperties[0][3].Name AFTER = 'TEZ2'
[...]
tdbfpTableFieldProperties[0][20].Name BEFORE = 'CGID'
tdbfpTableFieldProperties[0][20].Name AFTER = 'DGID'

tdbfpTableFieldProperties[1][0].Name BEFORE = 'DGID'
tdbfpTableFieldProperties[1][0].Name AFTER = 'TRV1'
tdbfpTableFieldProperties[1][1].Name BEFORE = 'TRV1'
tdbfpTableFieldProperties[1][1].Name AFTER = 'TEZ1'
tdbfpTableFieldProperties[1][2].Name BEFORE = 'TEZ1'
tdbfpTableFieldProperties[1][2].Name AFTER = 'TRV2'
tdbfpTableFieldProperties[1][3].Name BEFORE = 'TRV2'
tdbfpTableFieldProperties[1][3].Name AFTER = 'TEZ2'
[...]
tdbfpTableFieldProperties[1][67].Name BEFORE = 'TCTX'
tdbfpTableFieldProperties[1][67].Name AFTER = 'TAth'

tdbfpTableFieldProperties[2][0].Name BEFORE = 'TAth'
tdbfpTableFieldProperties[2][0].Name AFTER = 'TRV1'
tdbfpTableFieldProperties[2][1].Name BEFORE = 'TRV1'
tdbfpTableFieldProperties[2][1].Name AFTER = 'TEZ1'
tdbfpTableFieldProperties[2][2].Name BEFORE = 'TEZ1'
tdbfpTableFieldProperties[2][2].Name AFTER = 'TRV2'
tdbfpTableFieldProperties[2][3].Name BEFORE = 'TRV2'
tdbfpTableFieldProperties[2][3].Name AFTER = 'TEZ2'
[...]

I'm sure it's not a bug in the DLL, as I know of a Visual C# project that does the same thing I am trying to do in Python, with no problems. Any ideas what I might be doing wrong? Many thanks in advance!

`0x0EEDFADE` isn't a Windows error, per se. That's a Delphi exception code. It's using Windows SEH exceptions, which it's *leaking* (a C API shouldn't do that), and ctypes is re-raising it as a `WindowsError`. — Eryk Sun, Feb 21 '15 at 00:23
I see that the `FileName` parameter of `TDBOpen` should be a wide-character string. Are you possibly using `from __future__ import unicode_literals`? In Python 2, ctypes automatically converts between byte strings and wide-character strings. — Eryk Sun, Feb 21 '15 at 00:34
You probably shouldn't keep a private copy of the tables, since it won't reflect changes made in the actual tables. Also, the `Name` field in both structs needs to be allocated in Python. Do *not* use a Python 'immutable' string. It's wrong on principle, and in practice you see that the code object interns the string, leading to nonsense. Instead, I'd override `__init__(self, *args)` to set `self.Name = cast((c_wchar * 8)(), c_wchar_p);` `Structure.__init__(self, *args)`. — Eryk Sun, Feb 21 '15 at 00:53
No, I'm not using `... import unicode_literals`. My call to TDBOpen is just `intDBIndex = tdbaccessDLL.TDBOpen(r"C:\for Madden\current.ros")` Are you saying I should use unicode_literals, or no? From your comment that _ctypes automatically converts between byte strings and wide-character strings_, it sounds like you are saying that I don't need to worry about c_char_p vs. c_wchar_p. Is that so, or am I misunderstanding? — tuojiangosaurus, Feb 22 '15 at 19:42
Python 2 ctypes can only only convert between byte strings and wide-character strings when the type is known. Did you set `TDBOpen.argtypes`? Otherwise `r"C:\for Madden\current.ros"` gets passed as a byte string, and I'm afraid I have no idea how that's working unless either this is a lie: `function TDBOpen(const FileName: PWideChar): Integer; stdcall`, or magic. — Eryk Sun, Feb 22 '15 at 19:53
Hmm. Yes, I have set the argtypes (and restype) for each function I am calling, eg: `tdbaccessDLL.TDBOpen.argtypes = [c_char_p] tdbaccessDLL.TDBOpen.restype = c_int`. But that would seem to NOT tell ctypes to convert `r"C:\for Madden\current.ros"` to c_wchar_p. So, I'm leaning towards magic. :-P JK, I guess the docs are incorrect, which is sort of what I guessed when I couldn't get c_wchar_p to work as the type for Name in the tdbTableProperties struct. Oh well, more experimenting for me! — tuojiangosaurus, Feb 22 '15 at 20:11
Assuming the library uses `c_char_p` everywhere instead of `c_wchar_p`, change my struct `__init__` suggestion to set `self.Name = cast((c_char * 8)(), c_char_p)`. With this modification you need to remove the bogus `Name="blah"` passed to the constructor. Again, never pass a Python 'immutable' string as mutable data. The interpreter may intern the string (globally, or locally within the code object), as it does in this case. — Eryk Sun, Feb 22 '15 at 20:54
@eryksun That was exactly it. I finally got a chance to try out your suggestions, and initializing the Name field in the structs was the answer. (And yes, everything in the version of the DLL I am working with is byte string, not wide string, for some reason.) I would upvote your comment, but I can't see how (I think I don't have enough rep). Maybe make an answer out of it and I'll accept it? I assume I should be able to do that. — tuojiangosaurus, Feb 27 '15 at 16:14

score 2 · Answer 1 · answered Feb 20 '15 at 15:56

The first obvious error is that c_char_p is a pointer to 8 bit text. But the Delphi and VB.net structures clearly user 16 bit Unicode text. Replace c_char_p with c_wchar_p.

The next issue relates to the layout of the record. The Delphi and VB.net code don't agree. The Delphi record is packed, the VB.net struct is aligned. Your Python struct is aligned. You might try _pack_ = 1 in your ctypes structure to pack the record. It's anyone's guess as to how the record really is laid out.

We don't know for sure how large the enumerated types are. They could be 1, 2 or 4 bytes. We also don't have all the structure declarations to check, so there could be, and likely are, problems beyond what I have written here.

Yes, that first one would be obvious, and my first attempt at this used c_wchar_p for the Name fields. However, that resulted in the error `UnicodeEncodeError: 'charmap' codec can't encode characters in position 33-34: character maps to `, so I figured I'd try c_char_p just in case (despite the documentation) and when that seemed to work on the first call (to TDBTableGetProperties), I assumed it wasn't actually Unicode. But I'll admit my understanding of Unicode is far from complete, so I'll revisit that and see if I can't use c_wchar_p and resolve those errors instead. — tuojiangosaurus, Feb 22 '15 at 19:29

Why is this Delphi DLL throwing a WindowsError when called from Python?

1 Answers1