Py_InitializeEx(0) fails after Py_Finalize (after previous initialization)

Question

The question is about Python.h (embedded Python 3.10). Here in this minimal example where I Py_Initialize, import numpy and then Py_DecRef it. After that I Py_Finalize and repeat the process one more time:

#include <Python.h>
#include <memory>
#include <iostream>

class Pyth
{
public:
    explicit Pyth()
    {
        size_t len = 11;
        auto pname = Py_DecodeLocale("PythonCrash", &len);

        Py_SetProgramName(pname);

        Py_InitializeEx(0);

        auto *mname = PyUnicode_FromWideChar(L"numpy", 5);
        auto *pmod = PyImport_Import(mname);
        Py_DecRef(mname);
        if (pmod)
            Py_DecRef(pmod);
    }
    ~Pyth()
    {
        std::cerr << "Fin: " << Py_FinalizeEx() << std::endl;
    }
};

int main()
{
    int i = 2;
    while (i > 0) {
        std::cerr << "Go: " << i << std::endl;
        auto* ptr = new Pyth();
        std::cerr << "new pyth" << std::endl;
        delete ptr;
        std::cerr << "del pyth" << std::endl;
        --i;
    }

    return 0;
}

I'm getting an error after I try to finalize for the second time but stacks shows it's on second initialization attampt. The error does not appear if I won't do any work (that is, if I just initialize and finalize in a loop).

Here is the output:

17:27:41: Debugging /home/sms/build-PythonCrash-Desktop-Debug/PythonCrash ...
Go: 2
new pyth
Fin: 0
del pyth
Go: 1
new pyth
Fin: Traceback (most recent call last):
  File "/usr/lib64/python3.10/site-packages/numpy/__init__.py", line 150, in <module>
    from . import core
  File "/usr/lib64/python3.10/site-packages/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/usr/lib64/python3.10/site-packages/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/usr/lib64/python3.10/site-packages/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
SystemError: /builddir/build/BUILD/Python-3.10.4/Objects/structseq.c:481: bad argument to internal function
0
del pyth
17:27:42: Debugging of /home/sms/build-PythonCrash-Desktop-Debug/PythonCrash has finished with exit code 0.

Here is the stack:

1  PyArray_Item_INCREF
2  PyArray_FromScalar
3  gentype_nonzero_number.lto_priv
4  PyObject_IsTrue.part.0
5  _PyEval_EvalFrameDefault
6  _PyEval_Vector
7  _PyObject_FastCallDictTstate
8  slot_tp_init
9  type_call
10 _PyObject_MakeTpCall
11 _PyEval_EvalFrameDefault
12 _PyEval_Vector
13 _PyEval_EvalFrameDefault
14 _PyEval_Vector
15 method_vectorcall
16 _PyEval_EvalFrameDefault
17 _PyEval_Vector
18 method_vectorcall
19 _PyEval_EvalFrameDefault
20 _PyEval_Vector
21 method_vectorcall
22 _PyEval_EvalFrameDefault
23 _PyEval_Vector
24 _PyEval_EvalFrameDefault
25 _PyEval_Vector
26 _PyEval_EvalFrameDefault
27 _PyEval_Vector
28 _PyObject_VectorcallTstate.lto_priv.5
29 object_vacall
30 _PyObject_CallMethodIdObjArgs
31 PyImport_ImportModuleLevelObject
32 builtin___import__
33 cfunction_call
34 _PyObject_MakeTpCall
35 _PyObject_CallFunctionVa
36 PyObject_CallFunction
37 PyImport_Import
38 PyImport_ImportModule
39 _PyCodecRegistry_Init
40 _PyCodec_Lookup
41 config_get_codec_name
42 init_interp_main
43 pyinit_main
44 Py_InitializeFromConfig
45 Py_InitializeEx
46 Pyth::Pyth (main.cpp, 15)       
47 main (main.cpp, 36)

Related question: https://stackoverflow.com/q/7676314 – Artyer May 05 '22 at 16:22 — Artyer, May 05 '22 at 16:22

score -2 · Answer 1 · answered May 29 '22 at 22:05

-2

'len' is an internal function of python so you cannot do an assignment. try changing len to "o_locale_len" or, something, shorter but just not 'len'.

answered May 29 '22 at 22:05

user650150

15
3

1

This has *nothing* to do with the OP's issue. They're writing C/C++ level code. Even at the Python level, you can name things `len`, you just shouldn't to avoid confusion from the name-shadowing. – ShadowRanger May 29 '22 at 22:09
@ShadowRanger the python trace cites 'bad argument to internal function' which is how python would interpret any assignment to the len() method. testing the solution would be low effort. – user650150 May 29 '22 at 22:14
1

That is not *at all* how Python interprets assignment to the `len` method. You can assign to `len` with *zero* errors, it's not actually a mistake from Python's perspective. Bad argument to internal functions is a C-level runtime issue (that means exactly what it says; you passed something that violates the API contract to a C-level function) and it's *wholly* unrelated to assigning to `len`. A C-level variable named `len` has *nothing* to do with the Python level builtin function `len`; the existence of one does not affect the other in any way. – ShadowRanger May 29 '22 at 23:35
bool, len are primitives used for py object introspection; but assuming that @shad – user650150 May 30 '22 at 13:33
bool, len are primitives used for py object introspection; used lots of place for introspection but @ShadowRanger is correct, 'len' is still used in internal bytes objects like below, so this could be collision that C buildtime makes fatal: pycore_bytes_methods.h:extern PyObject* _Py_bytes_isspace(const char *cptr, Py_ssize_t len); – user650150 May 30 '22 at 14:11
Nope. I'm guessing you've never written C/C++ before; the `len` local variable there has no relevance whatsoever to the `len` parameter received by any other CPython API function (if it had such a problem, none of those CPython C-API functions could work together). This answer is not just unhelpful to the OP, it's wrong in every detail and should be deleted. – ShadowRanger May 30 '22 at 16:39

Py_InitializeEx(0) fails after Py_Finalize (after previous initialization)

1 Answers1