108

I have a script where I launch with popen a shell command. The problem is that the script doesn't wait until that popen command is finished and go continues right away.

om_points = os.popen(command, "w")
.....

How can I tell to my Python script to wait until the shell command has finished?

DennisLi
  • 3,915
  • 6
  • 30
  • 66
michele
  • 26,348
  • 30
  • 111
  • 168

7 Answers7

136

Depending on how you want to work your script you have two options. If you want the commands to block and not do anything while it is executing, you can just use subprocess.call.

#start and block until done
subprocess.call([data["om_points"], ">", diz['d']+"/points.xml"])

If you want to do things while it is executing or feed things into stdin, you can use communicate after the popen call.

#start and process things, then wait
p = subprocess.Popen([data["om_points"], ">", diz['d']+"/points.xml"])
print "Happens while running"
p.communicate() #now wait plus that you can send commands to process

As stated in the documentation, wait can deadlock, so communicate is advisable.

Amir Hossein Baghernezad
  • 3,616
  • 27
  • 28
unholysampler
  • 17,141
  • 7
  • 47
  • 64
  • 1
    Check out the docs on [subprocess.call](http://docs.python.org/library/subprocess.html#convenience-functions) – thornomad May 14 '10 at 23:29
  • While subprocess is preferred in many answers, it cannot handle space and quota within command very well. The above answer does not directly solve the os.popen question. – Chang Jul 19 '19 at 23:51
  • subprocess can be up to 2x slower than os system - https://bugs.python.org/issue37790 – MonsieurBeilto Mar 07 '20 at 23:37
  • 2
    `subprocess.run()` was added in Python 3.5 and is "The recommended approach to invoking subprocesses" – LoMaPh Sep 23 '20 at 03:50
44

You can you use subprocess to achieve this.

import subprocess

#This command could have multiple commands separated by a new line \n
some_command = "export PATH=$PATH://server.sample.mo/app/bin \n customupload abc.txt"

p = subprocess.Popen(some_command, stdout=subprocess.PIPE, shell=True)

(output, err) = p.communicate()  

#This makes the wait possible
p_status = p.wait()

#This will give you the output of the command being executed
print "Command output: " + output
Touchstone
  • 5,575
  • 7
  • 41
  • 48
24

Force popen to not continue until all output is read by doing:

os.popen(command).read()
Alberto
  • 396
  • 4
  • 11
  • 3
    I'm no python expert, but this seems to be the simplest solution that makes the fewest modifications to the original code. Any reason why this wouldn't be a good solution? – jdmcnair Feb 21 '20 at 02:51
  • @jdmcnair my guess is that if the command threw an error, this .read() method would break the parent process. But some other methods decouple errors in the child process from affecting the parent process. I'm guessing that if the child process hangs forever, .read() will wait forever to a result. – Marc Maxmeister Sep 03 '21 at 17:38
13

Let the command you are trying to pass be 

os.system('x')

then you covert it to a statement

t = os.system('x')

now the python will be waiting for the output from the commandline so that it could be assigned to the variable t.

Azsgy
  • 3,139
  • 2
  • 29
  • 40
Chidhambararajan NRM
  • 373
  • 3
  • 13
7

What you are looking for is the wait method.

Olivier Verdier
  • 46,998
  • 29
  • 98
  • 90
  • But if I type: om_points = os.popen(data["om_points"]+" > "+diz['d']+"/points.xml", "w").wait() I receive this error: Traceback (most recent call last): File "./model_job.py", line 77, in om_points = os.popen(data["om_points"]+" > "+diz['d']+"/points.xml", "w").wait() AttributeError: 'file' object has no attribute 'wait' What is the problem? Thanks again. – michele May 14 '10 at 20:24
  • 14
    You did not click on the link I provided. `wait` is a method of the `subprocess` class. – Olivier Verdier May 16 '10 at 18:13
  • 3
    wait can deadlock if the process writes to stdout and nobody reads it – ansgri Dec 19 '16 at 15:03
  • subprocess can be up to 2x slower than os system - bugs.python.org/issue37790 – MonsieurBeilto Mar 07 '20 at 23:38
  • 1
    @MonsieurBeilto `os.system` has been deprecated; it has been replaced with `subprocess` – eDonkey Jul 22 '22 at 07:37
  • Also, that bug (https://github.com/python/cpython/issues/81971) was closed for lack of reproducibility and insufficient information about the platform it was seen on. – Daira Hopwood Oct 19 '22 at 07:15
7

wait() works fine for me. The subprocesses p1, p2 and p3 are executed at the same. Therefore, all processes are done after 3 seconds. 

import subprocess

processes = []

p1 = subprocess.Popen("sleep 3", stdout=subprocess.PIPE, shell=True)
p2 = subprocess.Popen("sleep 3", stdout=subprocess.PIPE, shell=True)
p3 = subprocess.Popen("sleep 3", stdout=subprocess.PIPE, shell=True)

processes.append(p1)
processes.append(p2)
processes.append(p3)

for p in processes:
    if p.wait() != 0:
        print("There was an error")

print("all processed finished")
simibac
  • 7,672
  • 3
  • 36
  • 48
0

I think process.communicate() would be suitable for output having small size. For larger output it would not be the best approach.

Barsha Lamichhane
  • 1