C DLL crack when called from Python

Question

I've a DLL that embeds Python interpreter using the C/Python API. The DLL works fine if called one time, but if the DLL is called twice, the code cracks and my program catch memory error. The C code calling DLL is simple and the call to DLL function(Which calls Python interpreter) is done one time, if the second call(In the code) is not commented the code cracks and this happens only if "Numpy" is called in the Python code.

#include <stdio.h>
#include <conio.h>
#include <math.h>
#include <dll_simples.h>

int main() {
double in[] = { 4,2,5,4,2 };
double out[5] = {};
double a = 0;
double b = 0;


simuser(a,b,in,out);
//simuser(a, b, in, out); IF NOT COMMENTED -> ERROR

return 0;
 }

I've commented Py_Finalize() in the DLL as suggested here and this post here affirms that from Py_Finalize() docs"Some extensions may not work properly if their initialization routine is called more than once; this can happen if an application calls Py_Initialize() and Py_Finalize() more than once." So i'd like to know why this happens and if any other thing could be done except calls "Py_Finalize()" only one time in the last call to DLL.

Answer 1

Listing [Python 3.Docs]: ctypes - A foreign function library for Python.

dll00.h:

#pragma once

#if defined(_WIN32)
#  if defined DLL0_EXPORTS
#    define DLL00_EXPORT_API __declspec(dllexport)
#  else
#    define DLL00_EXPORT_API __declspec(dllimport)
#  endif
#else
#  define DLL00_EXPORT_API
#endif


#if defined(__cplusplus)
extern "C" {
#endif

DLL00_EXPORT_API int dll00Func00(double t, double delta, unsigned int size, const double *pIn, double *pOut);

#if defined(__cplusplus)
}
#endif

dll00.c:

#include <stdio.h>
#include "Python.h"
#define DLL0_EXPORTS
#include "dll00.h"

#define C_TAG "From C .dll"


int dll00Func00(double t, double delta, unsigned int size, const double *pIn, double *pOut) {
    int res = 0;
    printf("%s: in function\n", C_TAG);
    const int isInit = Py_IsInitialized();
    // Modify array calling Python functions
    if (!isInit) {
        printf("%s: initializing Python interpreter\n", C_TAG);
        Py_Initialize();
    }
    res = PyRun_SimpleString("print(\"From Python (within C .dll): test\")");
    for (unsigned  int i = 0; i < size; i++) {
        pOut[i] = pIn[i] * t + delta;
    }
    if (!isInit) {
        printf("%s: uninitializing Python interpreter\n", C_TAG);
        Py_Finalize();
    }
    return 0;
}

main00.c:

#include <stdio.h>

#include "dll00.h"
#define SIZE 4


int main() {
    int res = 0;
    double in[SIZE] = { 10.0, 11.0, 12.0, 13.0 };
    double out[SIZE] = { 0 };
    res = dll00Func00(2, 0.5, SIZE, in, out);
    printf("Output array:\n");
    for (unsigned int i = 0; i < SIZE; i++) {
        printf("%.03f ", out[i]);
    }
    printf("\n");
    return 0;
}

code00.py:

#!/usr/bin/env python

import sys
import ctypes as ct


DLL_NAME = "./dll00.dll"


def main(*argv):
    DblPtr = ct.POINTER(ct.c_double)
    size = 5
    DblArr = ct.c_double * size

    dll00 = ct.PyDLL(DLL_NAME)
    dll00Func00 = dll00.dll00Func00
    dll00Func00.argtypes = (ct.c_double, ct.c_double, ct.c_uint, DblPtr, DblPtr)
    dll00Func00.restype = ct.c_int

    in_arr = DblArr(*range(size))
    out_arr = DblArr()
    print("Output array:")
    for i in range(size):
        print("{:.3f}".format(out_arr[i]), end=" ")
    print("\n")
    res = dll00Func00(2, 2.5, size, in_arr, out_arr)
    print("Output array:")
    for i in range(size):
        print("{:.3f}".format(out_arr[i]), end=" ")
    print()



if __name__ == "__main__":
    print("Python {0:s} {1:d}bit on {2:s}\n".format(" ".join(item.strip() for item in sys.version.split("\n")), 64 if sys.maxsize > 0x100000000 else 32, sys.platform))
    main(*sys.argv[1:])
    print("\nDone.")

Notes:

• In Python, use ctypes.PyDLL as you're (indirectly) calling Python API functions
• In the .dll, use [Python 3.Docs]: Initialization, Finalization, and Threads - int Py_IsInitialized()
  • As a side note, the if test is not needed in Py_Initialize's case as Py_Initialize simply returns if the interpreter is already initialized (so I left it there just for consistency), but it is needed for Py_Finalize as one wouldn't want to uninitialize the interpreter while still in Python.
    So Py_Initialize / Py_Finalize pair doesn't work on "reference count" (every Py_Initialize call requires an Py_Finalize one)

• Calling Py_Initialize / Py_Finalize in the function, seems like an overkill (if the function is being called multiple times). I'd do 2 wrapper functions in the .dll and call:

  • one at the beginning
  • the other at the end

  of the (C) program

Output:

e:\Work\Dev\StackOverflow\q059937552>sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[prompt]> "c:\Install\pc032\Microsoft\VisualStudioCommunity\2017\VC\Auxiliary\Build\vcvarsall.bat" x64
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.19
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

[prompt]> dir /b
code00.py
dll00.c
dll00.h
main00.c

[prompt]> cl /nologo /MD /DDLL /I"c:\Install\pc064\Python\Python\03.07.06\include" dll00.c  /link /NOLOGO /DLL /OUT:dll00.dll /LIBPATH:"c:\Install\pc064\Python\Python\03.07.06\libs"
dll00.c
   Creating library dll00.lib and object dll00.exp

[prompt]> cl /nologo /MD /W0 main00.c  /link /NOLOGO /OUT:main00_064.exe dll00.lib
main00.c

[prompt]> dir /b
code00.py
dll00.c
dll00.dll
dll00.exp
dll00.h
dll00.lib
dll00.obj
main00.c
main00.obj
main00_064.exe

[prompt]> "e:\Work\Dev\VEnvs\py_pc064_03.07.06_test0\Scripts\python.exe" code00.py
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] 64bit on win32

Output array:
0.000 0.000 0.000 0.000 0.000

From C .dll: in function
From Python (within C .dll): test
Output array:
2.500 4.500 6.500 8.500 10.500

Done.

[prompt]> set PATH=%PATH%;c:\Install\pc064\Python\Python\03.07.06

[prompt]> main00_064.exe
From C .dll: in function
From C .dll: initializing Python interpreter
From Python (within C .dll): test
From C .dll: uninitializing Python interpreter
Output array:
20.500 22.500 24.500 26.500