Is embedded python Py_Finalize() blocking?

Question

I'm seeing intermittent crashing when I run large embedded python programs. My question is does the Py_Finalize() call block until all of the python interpreter is in a safe state before continuing? If it doesn't, how do I know when the interpreter has destroyed everything?

My current code looks like this:

Py_Initialize();
...
...
Py_Finalize(); // Unsure if this returns immediately or returns after completing all Finalizing actions

illumi · Accepted Answer · 2014-07-29T12:16:24.353

1

I don't think this is totally answering the question I originally asked, but, I have found a way to make the garbage collector do a better job when we I call Py_Finalize. That is to stop using static class variables in Python.

Old code:

class MyClass(object):
    a = {}
    def __init__(self):
        ...
    ...

New code:

class MyClass(object):
    def __init__(self):
        self.a = {}
        ...
...

edited Jul 29 '14 at 12:16

answered Jul 21 '14 at 13:03

illumi

274
5
17

1

This doesn't look like valid Python to me. You're defining a class as a function, and calling `__init__` rather than defining it. – Ruben Bakker Jul 21 '14 at 13:29
@RubenBakker Edited, the previous answer was rushed – illumi Jul 22 '14 at 13:14

Pablo Stark · Answer 2 · 2014-07-18T07:06:07.100

If I'm right calling Py_Finalize(); will clear the python interpreter (some exceptions are found on [1]).

I would suggest you to create a class for the python interpreter and to manually check all your tasks are finished before calling Py_Finalize();. In the projects where I have worked using the embedded python interpreter, this was suiting the best.

Hope it helps!

[Python Doc][1] https://docs.python.org/2/c-api/init.html

== EDIT ==

For Py_Finalize()

Bugs and caveats: The destruction of modules and objects in modules is done in random order; this may cause destructors (del() methods) to fail when they depend on other objects (even functions) or modules. Dynamically loaded extension modules loaded by Python are not unloaded. Small amounts of memory allocated by the Python interpreter may not be freed (if you find a leak, please report it). Memory tied up in circular references between objects is not freed. Some memory allocated by extension modules may not be freed. Some extensions may not work properly if their initialization routine is called more than once; this can happen if an application calls Py_Initialize and Py_Finalize more than once.

Seems like if your program is only calling Py_Initialize() and Py_Finalize() once, you might find some trouble (which I never did) and have some memory leak. However if you are only Initializing the python interpreter and performing tasks while your main program is running (I'm more familiar to this approach) you won't have many trouble.

How can this be reliable? According to the docs: `The destruction of modules and objects in modules is done in random order; this may cause destructors (__del__() methods) to fail when they depend on other objects (even functions) or modules` — illumi, Jul 16 '14 at 12:38
I'm afraid I'm not that familiar with the usage of ''PyEval_InitThreads();'' and Multithreading involving a python embedded interpreter, but hopefully there's a good starting point here in stackoverflow (at least I think so): http://stackoverflow.com/questions/15470367/pyeval-initthreads-in-python-3-how-when-to-call-it-the-saga-continues-ad-naus — Pablo Stark, Jul 17 '14 at 07:26
I'm going to delete the InitThreads from the question, I simply want to know about Finalize. — illumi, Jul 17 '14 at 13:22
I have just added a few lines to my answer, hope those are useful. — Pablo Stark, Jul 18 '14 at 07:06

Is embedded python Py_Finalize() blocking?

2 Answers2