How to embed a python code which sets flags in C++

Question

I couldn't find a way to embed a python code which sets flags in C++ code at https://docs.python.org/3.5/extending/embedding.html.

I have this python code test.py:

import tensorflow as tf

# Settings
flags = tf.app.flags
FLAGS = flags.FLAGS

#core params..
flags.DEFINE_string('model', 'gcn', 'model name')
flags.DEFINE_float('learning_rate', 0.01, 'initial learning rate')
flags.DEFINE_string("model_size", "small", "define model size")


def main(argv=None):
    print("Flags Set")
    print(FLAGS.learning_rate)

if __name__ == '__main__':
    tf.app.run()

when I execute the command:

python -m test --learning_rate 0.0002

The output is:

Flags Set
0.0002

How can I embed the above python code in C++ and invoke it?

Your first sentence is a bit strange as you link the exactly doc. page which explains how to do it. C++ code is usually compiled to binary (although it might be interpreted as well as I recently learnt). Python code is usually interpreted by a Python interpreter (although there might be compilers like e.g. [IronPython](https://de.wikipedia.org/wiki/IronPython)). Hence, this is what the doc. describes: Embedding a Python interpreter in a C or C++ program. The C++ code may also extend the Python by additional types, variables, and functions for interop. of Python and C++. — Scheff's Cat, Apr 08 '19 at 05:35
According to the doc's description, the embedded python code functions need to be invoked via ./executable . But when I executing my python code separately I want to give flag values (eg:--learning_rate 0.0002). I couldn't find a way to set those flags when I embed the python code in C++ — Anuradha Karunarathna, Apr 08 '19 at 06:02
For a start, have a look at this: [SO: How to access a Python global variable from C?](https://stackoverflow.com/q/285455/7478597). (I googled with [google "find a specific python variable c c++"](https://www.google.com/search?q=find+a+specific+python+variable+c+c%2B%2B) to find this.) I must admit, I needed a while to combine C++ and Python properly (and a while longer to get it stable) but I used exclusively Python doc. or google research to master it. With a bit patience, it's possible. ;-) — Scheff's Cat, Apr 08 '19 at 06:08
May be, I mis-understood your question. If you ask how to pass command line arguments via `PyRun_` functions -> I found this: [SO: Pass argument to PyRun_File(***)](https://stackoverflow.com/q/22381711/7478597) linking to this Blog article: [Passing Parameters in From C to Pyhton.](http://opensourcewithcool06.blogspot.com/2014/03/passing-parameters-in-from-c-to-pyhton.html). — Scheff's Cat, Apr 08 '19 at 06:19
https://docs.python.org/3.5/extending/embedding.html is about executing a function in python script not the python script itself. — Shoyeb Sheikh, Apr 08 '19 at 07:46
Some years ago I used [Boost.Python](https://www.boost.org/doc/libs/1_70_0/libs/python/doc/html/index.html) to add scripting ability to a C++ application. — wilx, Apr 23 '19 at 08:23
@wilx can you suggest me a good tutorial to follow to add scripting ability to c++ application using Boost.Python — Anuradha Karunarathna, Aug 02 '19 at 13:03

Shoyeb Sheikh · Accepted Answer · 2019-04-23T07:56:34.533

1

You have two options,

1. Simply execute the python script with system(),

system("python -m /path/to/test.py  --learning_rate 0.0002");

2. Use Python/C Api as follows,

#include <python3.6/Python.h>
#include<iostream>
using namespace std;

int main(int argc, char *argv[])
{
    FILE* file;
    wchar_t *program = Py_DecodeLocale(argv[0], NULL);
    wchar_t** _argv;
    for(int i=0; i<argc; i++){
        wchar_t *arg = Py_DecodeLocale(argv[i], NULL);
        _argv[i] = arg;
    }
    Py_SetProgramName(program);
    Py_Initialize();   
    PySys_SetArgv(argc, _argv);
    file = fopen("/path/to/test.py","r");
    PyRun_SimpleFile(file, "/path/to/test.py");
    Py_Finalize();
    return 0;
}

If you get an executable in a.out, you can run it like below,

./a.out --learning_rate 0.0002

Note:- I have Python.h in python3.6m folder and I used a flag -lpython3.6m for compilation.

edited Apr 23 '19 at 07:56

answered Apr 08 '19 at 10:14

Shoyeb Sheikh

2,659
2
10
19

Thanks a lot. I prefer the second option. However, I did a small change to the above code. I put Py_Initialize(); as the first line of the main method. Otherwise, I got ImportError: No module named 'tensorflow' – Anuradha Karunarathna Apr 08 '19 at 13:19
I would have tested it if I had tensorflow, glad it worked for you. – Shoyeb Sheikh Apr 08 '19 at 14:00
if I take this main method content into a separate function and invoke that function from two threads, I end up with a segmentation fault. Do you have an answer for that? – Anuradha Karunarathna Aug 02 '19 at 18:58

How to embed a python code which sets flags in C++

1 Answers1