Python XDECREF failing on 64bit mingw

Question

I am calling a python method from C using the following code:

#include "Python.h"

char *find_site(void)
{
    char *app_site_dir = 0;
    PyObject *module = 0;
    PyObject *result = 0;
    PyObject *module_dict = 0;
    PyObject *func = 0;

    module = PyImport_ImportModule((char *)"Site"); /* new ref */
    if (module == 0)
    {
    PyErr_Print();
    printf("Couldn't find python module Site\n");
    goto out;
    }
    printf("module = %p\n", module);
    module_dict = PyModule_GetDict(module); /* borrowed */
    if (module_dict == 0)
    {
    PyErr_Print();
    printf("Couldn't find read python module Site\n");
    goto out;
    }
    func = PyDict_GetItemString(module_dict, "find_site"); /* borrowed */
    if (func == 0)
    {
    PyErr_Print();
    printf("Couldn't find Site.find_site\n");
    goto out;
    }
    result = PyEval_CallObject(func, NULL); /* new ref */
    if (result == 0)
    {
    PyErr_Print();
    printf("Couldn't run Site.find_site\n");
    goto out;
    }
    else if (result == Py_None)
    {
    printf("Couldn't find site\n");
    goto out;
    }
    printf("result = %p\n", result);
    app_site_dir = PyString_AsString(result); /* borrowed */
    if (app_site_dir == 0)
    {
    PyErr_Print();
    printf("Couldn't read result from Site.find_site\n");
    goto out;
    }
    app_site_dir = strdup(app_site_dir); /* keep in our own memory */
    if (*app_site_dir)
    {
    printf("Found Site at %s\n", app_site_dir);
    }
    else
    printf("Could not find Site\n");
out:;
    printf("result = %p decref\n", result);
    Py_XDECREF(result);
    printf("module = %p module\n", module);
    Py_XDECREF(module);

    return app_site_dir;
}

int main(int argc, char **argv)
{
    Py_Initialize();
    char *site = find_site();
    printf("Site = %s\n", site);
    free(site);
}

the python code is

import os

def find_site():
    return os.path.abspath(".")

(in the full application, this is more complex, but this cut-down example demonstrates the problem)

this is cross compiled on linux with mxe.cc like this: i686-w64-mingw32.static-c++ -I/usr/local/opt/mxe.master/usr/x86_64-w64-mingw32.static/include/python2.7 -o py32.exe py.cc -lpython27

and it runs on Windows as expected (from the Ubuntu shell here)

module = 028BC710
result = 0283D6B0
Found Site at \\wsl$\Ubuntu\home\dl
result = 0283D6B0 decref
module = 028BC710 decref
Site = \\wsl$\Ubuntu\home\dl

but when compiled for 64-bit x86_64-w64-mingw32.static-c++ -I/usr/local/opt/mxe.master/usr/x86_64-w64-mingw32.static/include/python2.7 -o py64.exe py.cc -lpython27

it fails to run:

module = 0000000002750408
result = 0000000000E62EF0
Found Site at \\wsl$\Ubuntu\home\dl
result = 0000000000E62EF0 decref

the Python call is successful, the value is returned and printed, but the XDECREF of result fails and the program bombs out with no other output.

In both cases the libpython was made using the .dll copied from the target Windows machine and pexports and dlltool to create the .a: e.g

pexports-0.47/pexports python27.dll > python27.def
i686-w64-mingw32.static-dlltool  --dllname python27.dll --def python27.def --output-lib libpython2.7.a

where might my problem be?

... if the return value of the python is `return "C:/Test/Path"` it works. if you add `test2 = test` and `return test2` it fails. if you say `test2 = "".join(c for c in path)` and `return test2` it fails. if you set `path2 = "C:/Test/Path` and `return test2` it works — Dave Lawrence, Dec 06 '19 at 18:50
using `Py_REFCNT` [in the C code] shows a value of 2 for a `return "c:/test"` but a value of 1 a `return test` — Dave Lawrence, Dec 06 '19 at 19:57
by redefinining the Py_DECREF macro in my application: `#define Py_DECREF(op) do { if (--op->ob_refcnt == 0) fprintf(stderr, "DECREF %s %d %p %d %s %p\n", __FILE__, __LINE__, op, Py_SIZE(op), Py_TYPE(op)->tp_name,Py_TYPE(op)->tp_dealloc ); fflush(stderr); } while(0)` this outputs lines like this: `DECREF vardef_file.cc 1601 0000000009F3C728 0 generator 0000000000000000` which appears to be showing the tp_dealloc is a null pointer. — Dave Lawrence, Dec 11 '19 at 22:31

score 0 · Answer 1 · answered Dec 11 '19 at 23:41

0

found and fixed, pyconfig.h was incorrect for mingw 64 bit use and had wrong sizeof for Py_ssize_t.

see https://bugs.python.org/issue39001

answered Dec 11 '19 at 23:41

Dave Lawrence

1,281
12
17

Python XDECREF failing on 64bit mingw

1 Answers1

Linked