2

hello i have question in Python subprocess module

I want to interaction with child process (ex: child's stdin, stdout)

This is child Program's code

  1 #include <stdio.h>
  2 
  3 int main(){
  4         char buf[1024];
  5         printf("asdfsadfasff\n");
  6         scanf("%s",buf);
  7         printf("%s\n",buf);
  8 }

this code is very simple, print the string, wait input, and print input string

and next, the python code is below

  1 #!/usr/bin/python
  2 from subprocess import *
  3 
  4 fd = Popen(["./random"],stdin=PIPE,stdout=PIPE)
  5 print "pid = %d"%(fd.pid) # print pid
  6 result = ""
  7 result = fd.stdout.read(-1) # read
  8 print result
  9 fd.stdin.write("write write write!!!\n")
 10 result = fd.stdout.read(-1)
 11 print result

in this case, I expected this program will work well, but in line 7 (fd.stdout.read(-1) was blocked and never working

I changed read function's param (read(-1), read(1), read(), read(1024)) but everything do not work

But, when i give string in stdin first, program worked.

I think child's stdout buffer was not fill when the program is ended... this is just my opnion

Is there any solution about this problem?

Edit1. when I execute program which is print string first, and wait user's input such as "sudo su" This worked well that I expected

I cannot understand why this case work well

  1 from subprocess import *
  2 
  3 fd = Popen(["sudo","su"],stdin=PIPE,stdout=PIPE)
  4 
  5 print fd.stdout.read(-1)
  6 print fd.stdin.write("asdfasdfas\n")
Joonsung Kim
  • 246
  • 1
  • 3
  • 15
  • what happens if you `flush(stdout);` after your first `printf`? – mgilson May 15 '12 at 19:24
  • Apparently the C function is `fflush`...sorry, my C is a little rusty. – mgilson May 15 '12 at 19:30
  • 1 #include 2 3 int main(){ 4 char buf[10]; 5 printf("asdfsadfasff\n"); 6 fflush(stdout); 7 scanf("%s",buf); 8 printf("%s\n",buf); 9 } add fflush, but result is not changed, is there something mistake in timeing of call fflush? – Joonsung Kim May 15 '12 at 19:43
  • oh my... sorry to indent and line feed is ambigious – Joonsung Kim May 15 '12 at 19:44
  • @JoonSunKim without the line numbers, it's still valid C code ;). It looks like what I had envisioned though. You're also passing a string that is too long for your character buffer ( http://stackoverflow.com/a/2430310/748858 ). That could result in some funny behavior as well. – mgilson May 15 '12 at 19:48
  • @mgilson oh.. sorry to my example source code is wrong. but, buffer overflow is not issue in this problem. In my problem is that Popen.stdout.read is blocked – Joonsung Kim May 15 '12 at 20:33
  • @mgilson in source code, child program's buffer will be increased. I fixed post – Joonsung Kim May 15 '12 at 20:33

1 Answers1

5

The child process ./random is using stdio, which by default does "full buffering" rather than "line buffering" when the process's stdout is not a tty device. Since python runs the process via pipes (which are not tty devices) the program is buffering its output, and not printing it at the time of the scanf call. The buffer is flushed (written to stdout) only (a) when it fills up; (b) when explicitly flushed with an fflush call; or (c) at program exit.

You can modify the child process to do explicit fflush calls, or you can use a pseudo-tty from Python, e.g., with pexpect or sh. (Note that the sh module by default uses a pty for output only but this is usually sufficient.) Or, in cases like this where the program you're running only "looks interactive" (but is actually totally predictable), you can just run it with the known input sequence pre-provided:

fd = Popen(["./random"], stdin=PIPE, stdout=PIPE)
result = fd.communicate("write write write!!!\n")[0]

although then of course you'll get all the stdout input mixed together, and have to separate out the two lines it printed (in this particular case).

jfs
  • 399,953
  • 195
  • 994
  • 1,670
torek
  • 448,244
  • 59
  • 642
  • 775
  • thank you so much But In my case, scanf is never executed before get first output string in parent, (my problem => Program make seed randomly, and use this seed to get random number (20 integer) and print it. after than wait to get some number, then check input and generated seed number is same) In this case, I understand full buffering system can solve the problem...... – Joonsung Kim May 15 '12 at 19:50
  • Is there any flag or something that can change shell's stdio buffer system bull buffering to line buffering? – Joonsung Kim May 15 '12 at 19:51
  • In general there's no way to change some other program's buffering behavior. However, if you use `pexpect` (see link in answer), you *can* interact with a program that assumes it is talking to a user, because `pexpect` opens a pseudo-tty, just like any ordinary interactive user-commands window. That is, you're not changing the child process in any way, you're just running it under an actual interactive session. – torek May 15 '12 at 20:41
  • can you see the more question which is in Edit1. when I execute 'sudo su' why is worked well? – Joonsung Kim May 15 '12 at 20:44
  • Re the edit: `sudo` opens `/dev/tty` to get access to the person running the command, even in the presence of I/O redirection. (In other words it tries not to read the password from `stdin` at all.) However, your example still just hangs for me, after I enter the password. – torek May 16 '12 at 03:02
  • I've pointed the link to the latest `pexpect` version. Feel free to rollback. – jfs Dec 15 '13 at 01:37