0

I'm using this code for python multithreading scripts (got it from web):

class PyMainGuard
{
public:
PyMainGuard()
{
    Py_Initialize();
    PyEval_InitThreads();
    mGilState = PyGILState_Ensure();
    mThreadState = PyEval_SaveThread();
}

~PyMainGuard()
{
    PyEval_RestoreThread( mThreadState );
    PyGILState_Release( mGilState );
    Py_Finalize();
}

private:
PyGILState_STATE mGilState;
PyThreadState* mThreadState;
};

class PyThrGuard
{
public:
PyThrGuard()
{
    mMainGilState = PyGILState_Ensure();
    mOldThreadState = PyThreadState_Get();
    mNewThreadState = Py_NewInterpreter();
    PyThreadState_Swap( mNewThreadState );
    mSubThreadState = PyEval_SaveThread();
    mSubGilState = PyGILState_Ensure();
}

~PyThrGuard()
{
    PyGILState_Release( mSubGilState );
    PyEval_RestoreThread( mSubThreadState );
    Py_EndInterpreter( mNewThreadState );
    PyThreadState_Swap( mOldThreadState );
    PyGILState_Release( mMainGilState );
}

private:
PyGILState_STATE mMainGilState;
PyThreadState* mOldThreadState;
PyThreadState* mNewThreadState;
PyThreadState* mSubThreadState;
PyGILState_STATE mSubGilState;
};

In main thread I need to create PyMainGuard, and then in every single thread I have to create PyThrGuard - that creates there independent interpretator, that allows me to use threading without problem. But they still share same resources.

For example, if I will use this scrypts:

import win32evtlog
import time

server = "localhost"
logtype = "TestEventSource"
archfile = "C:\test.txt"

EVENTLOG_SUCCESS = 0

hWriteLog = win32evtlog.RegisterEventSource(None, logtype.decode('unicode-escape'))

strings = []

for x in range(0, 5):
    del strings[:]
    for y in range(0, 3):
        a = "Python run [%d] with part %d" % (x, y)
        strings.append(a.decode('unicode-escape'))
        time.sleep(2)
    win32evtlog.ReportEvent(hWriteLog, EVENTLOG_SUCCESS, 0, 515, None, strings, None)

win32evtlog.DeregisterEventSource(hWriteLog)

Sometimes I can get crash, sometimes there can be created events in windows log with more or less then 3 lines of text (i.e. different threads will write to one event, having same EventHandle, I guess).

Everything looks like my threads still works in same interpeter, not in different ones. Or there is a chance that win32evtlog is very thread unsafe, it loads just ones (not for every interpreter, as I want) and creates such problems.

Question is: how to achieve different interpreters for each thread?

Arkady
  • 2,084
  • 3
  • 27
  • 48
  • 1
    possible duplicate of [Multiple independent embedded Python Interpreters on multiple operating system threads invoked from C/C++ program](http://stackoverflow.com/questions/2968317/multiple-independent-embedded-python-interpreters-on-multiple-operating-system-t) – John Zwinck Mar 24 '14 at 12:47
  • Can you tell us why you want to have multiple threads in Python? This is a fairly well known no-no, but there are usually other solutions. – John Zwinck Mar 24 '14 at 12:50
  • I found solution, that I wrote above. I just don't understand what's wrong there. I choose to use multiple threads instead of multiple process because of non-POD data sharing between threads. Really I'm not well-experienced with cross-process data exchanging, but as I know, it's not easy to share non-POD data between processes, and it's simply to do it between threads (just giving pointer). – Arkady Mar 24 '14 at 13:08
  • For example, I have thread-safe queue with orders to python script. With multithreading I just need to take it from queue and give to script. With multiprocessing I need to pack from each non-POD message - POD data, send it to process, get POD-data, unpack it to non-POD message. That looks much harder. – Arkady Mar 24 '14 at 13:09
  • Have you seen multiprocessing.Queue? http://docs.python.org/2/library/multiprocessing.html#multiprocessing.Queue This will let you send objects between multiple processes pretty easily. You say "I found solution" but the solution you found crashes, so it isn't a solution at all. :) – John Zwinck Mar 24 '14 at 13:17
  • Thank you for that Queue. But my threads are not python threads, so I need to get something same at C++, if I will refuse using python in different threads. – Arkady Mar 24 '14 at 14:02
  • If your threads are not Python threads then you should simply release the GIL in each one of them, and not do Python things in those threads except when absolutely necessary, at which point you'll need to acquire the GIL again. – John Zwinck Mar 24 '14 at 14:17
  • GIL removes any benefits from multithreading, really scripts will be executed one after one. For example, if you have python function with sleep(5) print(hello) and use 5 python threads, then with GIL you will have "hello" at console 5 times every 5 second, last one - after 25 seconds after start. If you will use my code above - you will get 5 "hello" at console at one moment, after 5 seconds. Without any crushing. I just wonder that with my script above that works bad. – Arkady Mar 24 '14 at 14:34
  • I know that the GIL removes some benefits of multithreading, but not all. You can still do things in parallel outside of Python. Trying to run parallel Python interpreters in one process is probably not what you want. You could still use memory mapping or some other technique to share data between multiple processes. – John Zwinck Mar 24 '14 at 14:39

0 Answers0