-1

I am using subprocess of python27 lib to run a big python file(execute.py) from another python file(sample.py).

If I run the sample.py (which has subprocess statements) in windows command line,it is running properly and streaming the live output well.

But in the python GUI console,when I run the GUI python file(has same subprocess statements) the GUI window is Not responding for some minutes after some time the output is printing as whole(not streaming).

Here is the snippet:

cmdlist = ["python", "execute.py","name","xyz"]
proc = subprocess.Popen(cmdlist, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

for line in iter(proc.stdout.readline, ""):
        self.output.write(line)

self.output.write("\n Finished process\n")

Hitting my head for a week and could not find any solution so far.

MisterMiyagi
  • 44,374
  • 10
  • 104
  • 119
Nithya
  • 614
  • 2
  • 11
  • 28

2 Answers2

0

When you run script in command line: the output will send directly to the terminal. The Python GUI console(IDLE) base on tkinter tool kit so when you run the script: the output will send to the buffer. The GUI tool take a bit of time to show the buffer to screen. Therefore the time to show the output will be longer when running with command line. If your script print too much, the buffer will be overflow then the "Not responding" occurs.

Hong Phuc Nguyen
  • 36
  • 6
0

Do not set stdout or stderr if you want them to go to an outer (controlling) process.

cmdlist = ["python", "execute.py","name","xyz"]
proc = subprocess.Popen(cmdlist)
# do other stuff here if you like
self.output.write(
  "\n Finished process (%s)\n" % proc.wait()  # wait until process has finished; process will stream to stdout/stderr while we wait
)

If you set stdout or stderr to subprocess.PIPE your current (python) process creates a new pipe by which it communicates with the subprocess. Basically, your own process will buffer the subprocess' output.

Even if you don't explicitly buffer anything, python itself will do some buffering. This especially applies when you redirect stderr and write it to your own stdout - stdout is buffered by default, while stderr would not have been. Many programs also write to the same line and reset the cursor, thus doing proc.stdout.readline() will buffer, waiting for the line to finish.

Now, if you don't set stdout and stderr, the subprocess will inherit your stdout and stderr. This means it can write directly to the outer, controlling process.

MisterMiyagi
  • 44,374
  • 10
  • 104
  • 119
  • It is throwing Attribute error.` AttributeError: 'NoneType' object has no attribute 'readline' ` the file is running in the background(thread) – Nithya Jul 07 '16 at 08:51
  • @Nithya My bad. Of Course, you cannot do `proc.stdout.redline` when not redirecting `stdout`, since `proc.stdout` is not set then. I've fixed the corresponding section. – MisterMiyagi Jul 07 '16 at 09:14
  • still can i use the for loop in the `# do other stuff` section? – Nithya Jul 07 '16 at 09:30
  • @Nithya You cannot use `proc.stdout`, if that's what you mean. Are you attempting to `[tee`](https://en.wikipedia.org/wiki/Tee_(command))? If so, look at [this post](http://stackoverflow.com/a/2997533/5349916). Basically, you have to use `proc.stdout.read` instead of `proc.stdout.readline`. – MisterMiyagi Jul 07 '16 at 10:07
  • No.Not `tee`.I am trying to display the live output in wxpython GUI window. – Nithya Jul 07 '16 at 10:33
  • @Nithya I get that. The question is whether you *also* need the output in your python application or not. – MisterMiyagi Jul 07 '16 at 10:34
  • Partially correct.The output is printing in python application.The problem is while running the script the application window went to not responding state and the output is not streaming as live instead it is printed as whole.I nee it to live streaming as in windows command – Nithya Jul 07 '16 at 10:41