5

I'm writing an application that has an embedded Python interpreter, but can also run script files from within the application.

What I want to archive is to have a way to reset the interpreter or that it can execute a file/statement without affecting the global environment so you can run a script, edit the file and run the script again without finalizing python in between (or reset my interpreter).

I've tried to create a new interpreter but then sometimes it hangs the whole application (deadlock). Here is a small demo to see the problem.

Py_Initialize();
PyThreadState* pGlobalThreadState = PyThreadState_Get();
PyThreadState* pInterpreterThreadState = Py_NewInterpreter();
PyThreadState_Swap(pInterpreterThreadState);

PyRun_SimpleString("import PySide"); // Importing PySide deadlocks

Py_EndInterpreter(pInterpreterThreadState);
PyThreadState_Swap(pGlobalThreadState);

If I save GIL states it sometimes works, but that isn't supported together with Py_NewInterpreter according to documentation.

Using Python 3.4.

Is there any other solution to run commands in a separate environment or to fix the deadlock?

user1309971
  • 607
  • 1
  • 5
  • 11
  • 1
    Using multiple interpreters is incompatible with extensions that call `PyGILstate_Ensure`, such as PySide (check the backtrace). It's deadlocked in `PyEval_RestoreThread`, which was called because the thread-state saved in thread-local storage doesn't match the current thread-state. You can solve that, but there's still the problem that `PyGILstate_Ensure` only uses the single `autoInterpreterState` for new threads. See issues [10915](http://bugs.python.org/issue10915) and [15751](http://bugs.python.org/issue15751). – Eryk Sun May 16 '14 at 03:21
  • I see! What is the possible solutions for problems like this? Or is there any better solution to create a "clean" environment to execute in so the user doesn't have to unload modules or restart the application? – user1309971 May 19 '14 at 07:03
  • I called `PyEval_SaveThread`, and then in a new thread called `PyEval_RestoreThread`. In this new thread calling `Py_NewInterpreter` stores the new thread state to thread-local storage (TLS). Now when PySide calls `PyGILstate_Ensure` the current thread state matches the one saved in TLS and it doesn't deadlock. But this doesn't solve the real problem. When PySide creates a new thread that calls back into Python, `PyGILstate_Ensure` will only use the main interpreter. There are ideas to enhance this in the linked issues. Currently it's just not supported. – Eryk Sun May 19 '14 at 07:22
  • 3.4 source link for [`PyGILstate_Ensure`](http://hg.python.org/cpython/file/04f714765c13/Python/pystate.c#l763). – Eryk Sun May 19 '14 at 07:36
  • Thanks for the information!! Is there any good way if I only use the main interpreter and save/restore state? Like "save state, execute script, restore state of the environment (modules loaded, globals etc)" Maybe I can save the dictionaries and restore modules like this: http://stackoverflow.com/a/2794845/1309971 – user1309971 May 19 '14 at 07:56

0 Answers0