1

I run a Python script from a C++ program using PyRun_SimpleFile. I defined a custom module (using PyImport_AppendInittab, as done here), so it can be imported in my Python script and some C++ code gets executed by the Python script when functions from this module are used, this is done through a callback PyObject* MyFunction(PyObject *self, PyObject *args) being invoked by Python interpreter.

I want to know the current script file name and line number within the callback function being invoked.

I could not find any way to retrieve this. Is this possible?

Note: This question is definitely not a duplicate of How to get the caller's method name in the called method?. I'm trying to retrieve file name and line number from C++ code executing and later executed by a Python script.

jpo38
  • 20,821
  • 10
  • 70
  • 151
  • why not pass it as parameter? – 463035818_is_not_an_ai Oct 13 '20 at 11:34
  • @idclev463035818: Because the script calls many functions, and they have their own parameters, adding file name and line number would polute the original script. This is meant for debugging, so the idea is to make it work with my script without having to modify it. This is not an argument for a specific embedded function, it's some information I'd like to retrieve for any embedded function being called. – jpo38 Oct 13 '20 at 12:11
  • There is a way using `PyEval_GetFrame` and then digging through that object's members, but you might as well ask Python to do it for you: [How to get the caller's method name in the called method?](https://stackoverflow.com/questions/2654113/how-to-get-the-callers-method-name-in-the-called-method) – Botje Oct 13 '20 at 12:50
  • @Botje: I tried `PyEval_GetFrame()->f_lineno;`but this is always evaluated to 1...and also I coulf not find the path the the .py file being interpreted here. – jpo38 Oct 13 '20 at 13:29
  • I was not able to check, but you probably need the parent frame, as `PyEval_GetFrame` is the frame for your native function. – Botje Oct 13 '20 at 13:31
  • I just tried to iterate over `PyEval_GetGlobals()` hoping to find the information here...but that was unsuccessful. – jpo38 Oct 13 '20 at 13:51
  • look at traceback objects an their implementation in the cpython's source. you should follow f_back and get the f_lineno. ps: never keep a ref to these objects. – Szabolcs Dombi Oct 13 '20 at 14:25

1 Answers1

1

You will need PyTraceBack_Here.

You can take a look at a traceback object's implementation here

Here is an example printig the traceback created by PyTraceBack_Here

#include <Python.h>

PyObject * mymodule_meth_test(PyObject * self) {
    PyTraceBack_Here(PyEval_GetFrame());
    PyObject * exc;
    PyObject * val;
    PyObject * tb;
    PyErr_Fetch(&exc, &val, &tb);
    PyTraceBack_Print(tb, PySys_GetObject("stderr"));
    Py_RETURN_NONE;
}

PyMethodDef module_methods[] = {
    {"test", (PyCFunction)mymodule_meth_test, METH_NOARGS, NULL},
    {},
};

PyModuleDef module_def = {PyModuleDef_HEAD_INIT, "mymodule", NULL, -1, module_methods};

extern "C" PyObject * PyInit_mymodule() {
    PyObject * module = PyModule_Create(&module_def);
    return module;
}

From the tb object you should be able to extract the filename and line number. It is an ordinary PyObject you can pass it to a python script or inspect it.

Here is how to extract the values without taking care of the refcounts:

    int line = PyLong_AsLong(PyObject_GetAttrString(PyObject_GetAttrString(tb, "tb_frame"), "f_lineno"));
    const char * filename = PyUnicode_AsUTF8(PyObject_GetAttrString(PyObject_GetAttrString(PyObject_GetAttrString(tb, "tb_frame"), "f_code"), "co_filename"));
Szabolcs Dombi
  • 5,493
  • 3
  • 39
  • 71
  • 1
    Thanks. Could you please elaborate how to retrieve file name and line number from this? I tried to call `PyTraceBack_Here` on result of `PyEval_GetFrame` but this constantly leaves `f_lineno` set to 1. – jpo38 Oct 13 '20 at 14:53
  • changed the code, this code is missing error and refcount handling. – Szabolcs Dombi Oct 14 '20 at 06:31
  • THis works fine for line number. But file name is "C", while I execute my script through `PyRun_SimpleFile`. – jpo38 Oct 15 '20 at 06:14
  • probably you could then use `PyImport_ImportModule` instead of `PyRun_SimpleFile` – Szabolcs Dombi Oct 15 '20 at 10:06
  • Thing is I don't want to simply import the module, I need to run a full script. With your example, I'd like to run a myfile.py file containing `import mymodule\nmymodule.test()` and then within the `mymodule_meth_test` I need to know that the file being exscuted is `myfile.py` (my be different than the file I run if the file imported another one and is currently executing a function from this file) and current line within the file. – jpo38 Oct 15 '20 at 12:07
  • 1
    Looks like "C" means I'm currently executing some C code, I tried to use `PyFrame_GetBack` to get the caller frame info hoping to be back to the Python file itself, but that was unsuccessful (it returns NULL). – jpo38 Oct 15 '20 at 12:18
  • Hold on, this apparently works fine an reports myfile.py when I isolate it to a simple program but gives "C" when ran from my complex program. I'll dig more but maybe your answer is correct and works just fine. – jpo38 Oct 15 '20 at 12:35
  • 1
    OK, that actually works. It's just that I was passing `PyRun_SimpleFile` second parameter as UTF-8....and then, `filename` ended up being "C" instead of "C:\\....\\myfile.py"! – jpo38 Oct 15 '20 at 13:17
  • I can't make this work when file contains special characters. Please check this other post: https://stackoverflow.com/questions/64385781/python-embedded-how-to-pass-special-characters-to-pyrun-simplefile (different issue I believe) – jpo38 Oct 16 '20 at 08:44
  • https://stackoverflow.com/questions/7469296/is-there-any-built-in-function-that-convert-wstring-or-wchar-t-to-utf-8-in-linu – Szabolcs Dombi Oct 16 '20 at 09:07
  • Thanks for the tip. I could pass `std::wstring_convert>().to_bytes(Py_DecodeLocale(script.c_str(), NULL))` and now the script runs....but then `PyUnicode_AsUTF8(PyObject_GetAttrString(PyObject_GetAttrString(PyObject_GetAttrString(tb, "tb_frame"), "f_code"), "co_filename"))` returns `NULL`. I suspect `PyUnicode_AsUTF8 ` is not valid anymore now the string format changed? – jpo38 Oct 16 '20 at 09:47
  • use PyErr_Print() if any of the the api calls fail they set an error too. use the cpython source code from github it is really helpful while embedding. – Szabolcs Dombi Oct 16 '20 at 09:52