Python process input/output using pipes

Question

I'm writing a C++ program, which 'll run Python 3 interpreter as a child process, then connect two Linux anonymous pipes - one to it's stdout and stderr and the second to interpreter's stdin ; next communicate with it via these channels.

I need to run Python in interactive mode, i.e. pass one command to it using input pipe and wait for the answer on the output pipe. Everything 'd be fine, but it seems Python can run interactive mode only if it's stdout and stdin connected to tty's.

Python docs quotation:

The interpreter operates somewhat like the Unix shell: when called with standard input connected to a tty device, it reads and executes commands interactively; when called with a file name argument or with a file as standard input, it reads and executes a script from that file.

Indeed when I'm running interpreter with pipes instead of tty, I can't see anything in response pipe, after the command was sent.

So - can I workaround such behaviour by some way and make python3 interpreter works exactly the way it was started from terminal by user?

Again, the problem in a nutshell:

I need to integrate Python to my C++ server app, to allow clients execute python commands. Embedding interpreter into server looks like a bad idea, mainly for reasons of safety ( users can damage server or it's data, besides server is running with some privileges I won't wanna grant to users ).

Another possible solution is to use interpreter in CLI way ( command mode). The main problem is - then I need import some modules and pre-execute some code to provide my server environment and some API to users. It 'll be too heavy to do it before calling interpreter with every command ( these actions are quite complex, including network connection establishing )

So, running interpreter in a separate process and server communicating with it using IPC mechanisms seems not so bad idea.

Anyway, I'll be glad to take a look at your suggestions If you have any.

Why do you need Python to run interactively? Can you not just use `python -c` to run a snippet non-interactively? — John Kugelman, Sep 10 '18 at 14:56
@JohnKugelman just need pre-import some python modules and do some stuff for user. It will be to heavy to do it every time running a command — Uroboros, Sep 10 '18 at 15:17
I'm still not following why you need it to be interactive. Please give me context about what you're trying to do with Python. I suspect this is an [XY problem](http://xyproblem.info/). — John Kugelman, Sep 10 '18 at 16:47
@JohnKugelman I've augmented the question, take a look for context please — Uroboros, Sep 13 '18 at 12:08

score 2 · Answer 1 · answered Sep 10 '18 at 14:50

2

You may use python in cli way, each time run the command user has entered, or archive them in a file and run per line entered. By this way, you can use it's output more simply. Hope to be useful.

answered Sep 10 '18 at 14:50

Jafari MM

57
1
7

Unfortunately, i need pre-import some python modules and do some stuff for user before execute his commands. It will be to heavy to do it every time running a command – Uroboros Sep 10 '18 at 15:18

Jim Janney · Answer 2 · 2018-09-11T01:56:37.160

If you really need the child process to think it's talking to a terminal, you can do that with a pseudo-terminal or pty. There are two different APIs, one from BSD Unix and one from System V, and Linux supports both. See What do pty and tty mean? or look at the man pages for forkpty or posix_openpt.

However, in this case my guess is that the python interpreter is block buffering its output. You could test this by invoking it with -u to disable buffering on stdin/stdout.

score 1 · Answer 3 · answered Sep 13 '18 at 11:29

You are doing something really weird and unusual. The interactive shell is intended for interactive use, with a human typing commands. It's not designed for interactive scripting where the input comes from another program.

With that said, yes you can do this, though I really wouldn't recommend it. What you want to do is run a script that reads stdin for commands and then call exec() method.

A simple version might look like this:

while True:
    lines = []
    while True:
        line = input()
        if line == "":
            exec("\n".join(lines))
            break
        else:
            lines.append(line)

The above program behaves similarly to the python shell when receiving multi-line input, in that interactive commands are buffered and not executed until there is a consecutive empty newlines.

For scripting/programmatic usage, you probably want a block marker that is a bit more robust than double newlines. For example, you may want to break command blocks with null character instead so that you can safely execute scripts that contains empty lines.

FWIW, I agree with John Kugelman that this looks like the wrong question to ask anyway. There are likely a better and easier way to do your actual problem than doing something like this. What is it that you are really trying to do?

I've augmented the question, take a look, please – Uroboros Sep 13 '18 at 12:07 — Uroboros, Sep 13 '18 at 12:07

Python process input/output using pipes

3 Answers3