2

I'm trying to write to stdin and read from stdout ( and stderr ) from an external program, without changing the code.

I've tried using named pipes, but stdout doesn't show until the program is terminated and stdin only works on the first input( then cin is null ).

i've tried using /proc/[pid]/fd but that only writes and reads from the terminal and not the program.

i've tried writing a character device file for this and it worked, but only one program at a time ( this needs to work for multiple programs at a time ).

at this point, to my knowledge, I could write the driver that worked to multiplex the io across multiple programs but I don't think that's the "right" solution.

the main purpose of this is to view a feed of a program through a web interface. I'm sure there has to be someway to do this. is there anything I haven't tried that's been done before?

tay10r
  • 4,234
  • 2
  • 24
  • 44
  • This can help: http://stackoverflow.com/questions/43116/how-can-i-run-an-external-program-from-c-and-parse-its-output If you need to read stdout and stderr of the process, you can execute it with redirecting stderr to stdout like this popen example: "popen("/bin/ls /etc/ 2>&1", "r");" – JustAnotherCurious Mar 20 '13 at 05:47

1 Answers1

5

The typical way of doing this is:

  1. Create anonymous pipes (not named pipes) with the pipe(2) system call for the new process's standard streams
  2. Call fork(2) to spawn the child process
  3. close(2) the appropriate ends of the pipes in both the parent and the child (e.g. for the stdin pipe, close the read end in the parent and close the write end in the child; vice-versa for the stdout and stderr pipes)
  4. Use dup2(2) in the child to copy the pipe file descriptors onto file descriptors 0, 1, and 2, and then close(2) the remaining old descriptors
  5. exec(3) the external application in the child process
  6. In the parent process, simultaneously write to the child's stdin pipe and read from the child's stdout and stderr pipes. However, depending on how the child behaves, this can easily lead to deadlock if you're not careful. One way to avoid deadlock is to spawn separate threads to handle each of the 3 streams; another way is to use the select(2) system call to wait until one of the streams can be read from/written to without blocking, and then process that stream.

Even if you do this all correctly, you may still not see your program's output right away. This is typically due to buffering stdout. Normally, when stdout is going to a terminal, it's line-buffered—it gets flushed after every newline gets written. But when stdout is a pipe (or anything else that's not a terminal, like a file or a socket), it's fully buffered, and it only gets written to when the program has outputted a full buffer's worth of data (e.g. 4 KB).

Many programs have command line options to change their buffering behavior. For example, grep(1) has the --line-buffered flag to force it to line-buffer its output even when stdout isn't a terminal. If your external program has such an option, you should probably use it. If not, it's still possible to change the buffering behavior, but you have to use some sneaky tricks—see this question and this question for how to do that.

Community
  • 1
  • 1
Adam Rosenfield
  • 390,455
  • 97
  • 512
  • 589
  • 1
    I would suggest to use `poll(2)` instead of `select(2)`. And `exec(3)` could actually be `execve(2)` which `exec(3)` is calling. A good book explaining that is http://advancedlinuxprogramming.com/ – Basile Starynkevitch Mar 20 '13 at 06:14
  • @Basile: Yes, `poll(2)` also works, the main advantage being that it takes O(number of file descriptors) time instead of (largest file descriptor) time. And of course `execve(2)` works, I just suggested `exec(3)` since it's more flexible (you can specify the arguments as an array or a direct function arguments and you can have it search `$PATH` for you). – Adam Rosenfield Mar 20 '13 at 15:27
  • If anyone is still reading this post, I did get this working and it does require modifying the buffering mechanism, as Adam has said. By setting stdbuf -o L, stdout becomes line buffered, which outputs the data before the end of the program. – tay10r Mar 21 '13 at 15:27