input to C++ executable python subprocess

Question

I have a C++ executable which has the following lines of code in it

/* Do some calculations */
.
.
for (int i=0; i<someNumber; i++){
   int inputData;
   std::cin >> inputData;
   std::cout<<"The data sent from Python is :: "<<inputData<<std::endl;
   .
   .
   /* Do some more calculations with inputData */
}

and this is called in a loop. I want to call this executable in python subprocess like

p = Popen(['./executable'], shell=True, stdout=PIPE, stderr=PIPE, stdin=PIPE)

I could get the output from the executable using

p.server.stdout.read()

But I am not able to send data (integers) from python using

p.stdin.write(b'35')

Since cin is called in a loop, the stdin.write should also be called multiple times (in a loop). Is this above possible .. ?

Any hints and suggestion how I could do it ? Thanks in advance.

omg ... i hope your calculations are very expensive other wise this is probably faster to just do in python or just do in C ... if they are expensive you should look at compiling your c++ into a dll or even a python library ... (and certainly take the user input out of the c code if you can) — Joran Beasley, Sep 14 '15 at 17:04
@JoranBeasley yes my computations are expensive to do them in python. That is the reason I chose to do it like this. Also I need to communicate small data with python. — AdityaG, Sep 14 '15 at 19:46

score 16 · Accepted Answer · edited May 23 '17 at 12:33

16

Here's minimalistic example of how to call a C++ executable from Python, and communicate with it from Python.

1) Note that you must add \n when writing to input stream (i.e. stdin) of the subprocess (just like you would hit Rtn if running the program manually).

2) Also note the flushing of the streams, so that the receiving program doesn't get stuck waiting for the whole buffer to fill before printing the result.

3) And if running Python 3, be sure to convert the streaming value from string to bytes (see https://stackoverflow.com/a/5471351/1510289).

Python:

from subprocess import Popen, PIPE

p = Popen(['a.out'], shell=True, stdout=PIPE, stdin=PIPE)
for ii in range(10):
    value = str(ii) + '\n'
    #value = bytes(value, 'UTF-8')  # Needed in Python 3.
    p.stdin.write(value)
    p.stdin.flush()
    result = p.stdout.readline().strip()
    print(result)

C++:

#include <iostream>

int main(){
    for( int ii=0; ii<10; ++ii ){
        int input;
        std::cin >> input;
        std::cout << input*2 << std::endl;
        std::cout.flush();
    }
}

Output of running Python:

edited May 23 '17 at 12:33

Community

1
1

answered Sep 14 '15 at 17:38

Velimir Mlaker

10,664
4
46
58

thank you very much for your help. But unfortunately I get error saying `TypeError: 'str' does not support the buffer interface` .... – AdityaG Sep 14 '15 at 19:44
1

@AdityaG just change to `p.stdin.write(b'35\n')` ... that use bytes instead of str (@Velimir is using str , maybe different python versions) – Joran Beasley Sep 14 '15 at 19:46
@Joran Beasley, you're right. AdityaG, you must be running Python 3. In that case, you must convert the string to bytes. I added a comment that does just that. – Velimir Mlaker Sep 14 '15 at 20:05
@Velimir ... one more small question, do you think we can send numpy array like that ..? (I will experiment but am just curious). – AdityaG Sep 14 '15 at 20:33
Well, using this methodology you can stream anything you want from Python, so long as your C++ executable properly parses standard input. But ideally you'd call your C++ computation routines directly in Python, by wrapping your C++ code (e.g. using SWIG) and generating Python callable functions -- creating subprocesses and streaming bytes is not optimal. – Velimir Mlaker Sep 14 '15 at 22:09

input to C++ executable python subprocess

1 Answers1

Linked