Context and formulation of a question
Hello everyone. I have a cpp program that calls python script multiple times in a loop. Everything works fine unless I add Py_Finalize() at the end of the function. In that case at the second iteration of the loop the program will try to extract value from PyObject* result variable which fro some reason appears to be nullptr in debugger at that point and then terminate itself.
Py_REFCNT() outputs consistent results for all objects throughout multiple executions of a function in the loop. By that I mean that reference count for all Pu_Object stays the same in each execution. You can spectate this by uncommenting std::cout lines in get_play_mrl(). However if you uncomment Py_Finalize() at the second iteration PyUnicode_AsWideCharString() will attempt to access result which at that moment would have no references and would be nullptr thus crashing the program.
Question: why Py_Finalize() causes this? How does result become nullptr if it is declared in the same function?
Side question 1: Can something bad happen if I call Py_Initialize() in a loop without matching each with Py_Finalize()?
Side question 2: I've noticed that some objects after being declared in code only once have more than single reference. Why it happens? Is Py_SET_REFCNT() the best way to dereference objects in such cases?
Python script that is being called from C++ code
#!/usr/bin/python3
import pafy
import youtube_dl
def get_play_url(url):
song = pafy.new(url)
duration = song.length
audiostreams = song.audiostreams
best = song.getbest()
play_url = best.url
return play_url
C++ code
// all commented lines besides one are used for debugging purposes
std::string get_play_mrl (
const std::string mrl
)
{
Py_Initialize();
PyObject* py_module_name = PyUnicode_FromString((char*)"get_str");
PyObject* py_module = PyImport_Import( py_module_name );
if (!py_module)
{
std::cout << "Error importing module.\n";
return "Error";
}
PyObject* function = PyObject_GetAttrString(py_module, (char*)"get_play_url");
const char* mrl_c = mrl.c_str();
PyObject* args = PyTuple_Pack(1, PyUnicode_FromString(mrl_c));
PyObject* result = PyObject_CallObject(function, args);
//std::cout << "Module: " << Py_REFCNT(py_module) << "\nFunction: " << Py_REFCNT(function) << "\nResult: " << Py_REFCNT(result) << "\n";
Py_ssize_t* size = nullptr;
wchar_t* play_mrl_wchar = PyUnicode_AsWideCharString(result, size);
std::wstring ws( play_mrl_wchar );
std::string play_mrl_str( ws.begin(), ws.end() );
delete [] play_mrl_wchar; // btw am I freeing memory in correct way here?
Py_XDECREF(py_module_name);
Py_XDECREF(py_module);
Py_XDECREF(function);
Py_XDECREF(args);
Py_XDECREF(result);
// uncommenting this will result in program crash at the second iteration of the loop
//Py_Finalize();
//std::cout << "Module: " << Py_REFCNT(py_module) << "\nFunction: " << Py_REFCNT(function) << "\nResult: " << Py_REFCNT(result) << "\n";
return play_mrl_str;
}
int main()
{
std::vector<std::string> url_vec = {/* Some links to short YouTube videos */};
for (auto a : url_vec)
{
std::string mrl = get_play_mrl(a);
// doing something with mrl...
}
return 0;
}
CMake file for building a project in case someone needs it
cmake_minimum_required(VERSION 3.18)
project(
py_to_cxx
LANGUAGES CXX C
)
set(PYTHON_HEADERS /usr/include/python3.9 )
set(PYTHON_STATIC /usr/lib64 )
add_library(python_embeded SHARED IMPORTED GLOBAL)
set_target_properties(python_embeded PROPERTIES
IMPORTED_LOCATION ${PYTHON_STATIC}/libpython3.9.so
INTERFACE_INCLUDE_DIRECTORIES ${PYTHON_HEADERS}
LINKER_LANGUAGE C
)
add_executable(py_to_cxx main.cpp)
target_link_libraries(py_to_cxx PRIVATE python_embeded)
