communicate() in a loop fails

Question

from subprocess import Popen, PIPE
import glob
import time

file_list=glob.glob(r'C:\Tony\files\*.txt')
numFiles = len(file_list)

p = Popen([r"C:\Tony\prog.bat"], stdin=PIPE)

start_time = time.time()

for x in range(0,numFiles):
    command = "run {" + file_list[x] + "} "
    p.communicate(input=command)[0]

print("--- %s seconds ---" % (time.time() - start_time))

I am starting a program by triggering the .bat using Popen. My program knows to take commands like run {path to .txt file} and it does things.

This works fine for the first iteration. But then I get the following error:

Traceback (most recent call last):
  File "run_tests.py", line 14, in <module>
    p.communicate(input=command)[0]
  File "c:\python27\lib\subprocess.py", line 465, in communicate
    self.stdin.write(input)
ValueError: I/O operation on closed file

Why is the file getting closed?

'Wait for process to terminate' (from doc) save the results in a variable. else each time you'll call it, it will wait for a closed process to end...which it can't do anymore.. if you are trying to use it for different files then you need to call a new process for each call — Frayal, Nov 26 '18 at 13:22
@Alexis I am not sure I understand what you mean. The process is still running in the background. I want to keep sending it commands. — Tony Tannous, Nov 26 '18 at 13:23
p.communicate will wait for it to end. it is no longer running. you can check the status — Frayal, Nov 26 '18 at 13:24
Is there a reason you need an answer for Python 2? The currently supported and recommended version of the language is Python 3. — tripleee, Nov 26 '18 at 13:29
@tripleee I have no permission to update python to version `3.x`. I am working on a remote host. — Tony Tannous, Nov 26 '18 at 13:30
With a modern Python you would use something like `subprocess.run(command, input=string)` and avoid the whole ordeal of dealing with raw `Popen()`. See also https://stackoverflow.com/a/51950538/874188 — tripleee, Nov 26 '18 at 13:30

Crazy · Accepted Answer · 2018-11-26T13:34:49.777

2

Please read documentation Popen.communicate

It wait for process to terminate.

Do not use it in loop.

You may use:

import subprocess

process = subprocess.Popen(["grep", "hello"],
                           stdin=subprocess.PIPE, 
                           stdout=subprocess.PIPE)

process.stdin.write("hello\nhello world\nhella")

print process.communicate()[0]

process.stdin.close()

edited Nov 26 '18 at 13:34

answered Nov 26 '18 at 13:19

Crazy

324
2
8

Or if you do use it in a loop, you'll have to `Popen` it each iteration. – Scott Hunter Nov 26 '18 at 13:29
@ScottHunter can this be done without having to Popen everytime? – Tony Tannous Nov 26 '18 at 13:31
You might want to pass the input as the `input=` argument to `communicate()` like the OP tried. Separately printing to stdin doesn't seem like an improvement; the only necessary change is to concatenate the input elements into a single string. – tripleee Nov 26 '18 at 13:39
(Though it's unclear what the OP hoped for the `[0]` to do; they are simply discarding this value anyway.) – tripleee Nov 26 '18 at 13:39
Concatenating the whole strings indeed works. Not what I hoped for but better than nothing. – Tony Tannous Nov 26 '18 at 13:51

communicate() in a loop fails

1 Answers1