Error when Importing keras in embedded python in C

Question

I'm trying to embed python in my C application. I download the package in python official website and manage to do a simple Hello World.

Now I want to go deeper and use some libraries of python like numpy, keras, tensorflow...

I'm working with Python 3.5.4, I installed all the needed package on my PC with pip3 :

pip3 install keras
pip3 install tensorflow
...

then I created my script and launch it in python environment, it works fine :

Python:

# Importing the libraries
#
import numpy as np
import pandas as pd

dataset2 = pd.read_csv('I:\RNA\dataset19.csv')
X_test = dataset2.iloc[:, 0:228].values
y_test = dataset2.iloc[:, 228].values

# 2.
import pickle
sc = pickle.load(open('I:\RNA\isVerb_sc', 'rb'))
X_test = sc.transform(X_test)

# 3. 
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout

classifier = Sequential()

classifier.add(Dense(units = 114, kernel_initializer = 'uniform', activation = 'relu', input_dim = 228))
classifier.add(Dropout(p = 0.3))

classifier.add(Dense(units = 114, kernel_initializer = 'uniform', activation = 'relu'))
classifier.add(Dropout(p = 0.3))

classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))

classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

classifier.load_weights('I:\RNA\isVerb_weights.h5')

y_pred = classifier.predict(X_test)
y_pred1 = (y_pred > 0.5)

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred1)

But when I execute the same script in a C environment with embed python it didn't work :

At first, I execute my script directly with PyRun_SimpleFile with no luck, so I sliced it in multiple instructions with PyRun_SimpleString to detect the problem :

C:

result = PyRun_SimpleString("import numpy as np"); // result = 0 (ok)
result = PyRun_SimpleString("import pandas as pd"); // result = 0 (ok)
...
result = PyRun_SimpleString("import pickle"); // result = 0 (ok)
... (all insctruction above works)
result = PyRun_SimpleString("import keras"); // result = -1 !!
... (all under this failed)

but there is not a single stack trace about this error, I tried this but I just got :

"Here's the output: (null)"

My initialization of Python in C seems correct since others libraries import fine :

    // Python
    wchar_t *stdProgramName = L"I:\\LIBs\\cpython354";
    Py_SetProgramName(stdProgramName);

    wchar_t *stdPythonHome = L"I:\\LIBs\\cpython354";
    Py_SetPythonHome(stdPythonHome);

    wchar_t *stdlib = L"I:\\LIBs\\cpython354;I:\\LIBs\\cpython354\\Lib\\python35.zip;I:\\LIBs\\cpython354\\Lib;I:\\LIBs\\cpython354\\DLLs;I:\\LIBs\\cpython354\\Lib\\site-packages";
    Py_SetPath(stdlib);

    // Initialize Python 
    Py_Initialize();

When inside a Python cmd, the line import keras take some time (3sec) but works (a warning but I found no harm around it) :

>>> import keras
I:\LIBs\cpython354\lib\site-packages\h5py\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
>>>

I'm at loss now, I don't know where to look at since there is no stack trace.

score 0 · Accepted Answer · answered Mar 07 '18 at 10:14

it seems like when you import keras, it executes this line :

sys.stderr.write('Using TensorFlow backend.\n')

or sys.stderr was not defined in python embedded on windows

A simple correction is to define sys.stderr, for example :

import sys
class CatchOutErr:
    def __init__(self):
        self.value = ''
    def write(self, txt):
       self.value += txt

catchOutErr = CatchOutErr()
sys.stderr = catchOutErr

Error when Importing keras in embedded python in C

1 Answers1