Importing Python module in C++ not in main

Question

I want to use a Python module within C++. In all examples I find (doc, SO1, SO2) they do things like Py_Initialize() and Py_FinalizeEx(), among other things, within the main function. In my application, however, I am writing a small part of a larger system and have no access to main. I am just writing a function that will be called many times somewhere in the program. What would be the way of doing all the initialization and finalization of the Python stuff in this case? I guess I could do everything within my function, but initializing the interpreter, importing, etc. every time the function is called will probably become very slow. I am not very experienced in C++, not sure if something like the following pseudocode is possible:

void my_func(void) {
  if (my_func_is_called_for_the_first_time()) {
    Py_Initialize();
    module = load_python_module();
    etc_related_to_init_stuff();
  }
  if (exiting_program()) { // Don't know how this would work, but you get the idea... Something like the `atexit` functionality from Python.
    clean_everything_related_to_python_and_the_module();
    Py_FinalizeEx();
  }
  // Actual code of my function using the package goes here...
}

Edit 1

Pseudocode:

#include <Python.h>

class MyPackage:
  def constructor:
    Py_Initialize();
    self.module = load_python_module();
    // Do other things related to the initialization here.
  
  def destructor:
    Py_FinalizeEx();

package = MyPackage(); // This is now global

void my_func(void) {
  blah();
  package.whatever();
}

What parts of implementing this have you tried, and what parts do you need help with? — Scott Hunter, Apr 13 '23 at 15:31
So far I only have the pseudocode implementation. I need help in running all things related to initialization only the first time the function is called (or somehow globally?) so it does not happen every time the function is called, and then everything related to finalization when the program ends, instead of every time the function is called. — user171780, Apr 13 '23 at 15:35
I could do everything each time the function is called, but does not sound like a good idea... — user171780, Apr 13 '23 at 15:35
What is preventing you from writing `my_func_is_called_for_the_first_time`? — Scott Hunter, Apr 13 '23 at 15:37
Another option would be to put that code outside my function, so that it is global. But I remember my C++ professor telling me never to do this kind of stuff. I have done this kind of things in small Arduino projects but don't know if it is a good practice for more serious stuff. — user171780, Apr 13 '23 at 15:37
If I do that anyhow the `module` object in my code will be lost, not available in future calls. So somehow I need to make it persist. — user171780, Apr 13 '23 at 15:38

score 2 · Answer 1 · answered Apr 13 '23 at 15:57

This is the perfect use-case for a static local variable:

struct PythonInitializer
{
    PyObject* module;
    PythonInitializer()
    {
        Py_Initialize();
        module = load_python_module();
    }

    ~PythonInitializer()
    {
        clean_everything_related_to_python_and_the_module();
        Py_FinalizeEx();
    }
};

void my_func()
{
    static PythonInitializer python;

    // use python.module
}

The trick here is that static local variables are constructed the first time a function is called and their lifetime extends to the end of the program. So a PythonInitizlizer object will be constructed the first time my_func is called, which will initialize the python interpreter. The PythonInitilizer's destructor will get called when the program exits (cleanly) and clean up the python interpreter.

Note that this will not play well with other users of python in your program. If another function calls Py_FinalizeEx then the interpreter will get unloaded out from under you. If there are other users of python in your application you should add a Py_IsInitialized() check to your function to re-load python if someone else unloads it or move initialization and cleanup somewhere else.

What would be the way to make it "multi user friendly"? Is there a way to instantiate multiple interpreters, or instances of Python, such that I can instantiate my own completely separate from others to avoid issues? — user171780, Apr 22 '23 at 16:43

Importing Python module in C++ not in main

Edit 1

1 Answers1