83

If I run echo a; echo b in bash the result will be that both commands are run. However if I use subprocess then the first command is run, printing out the whole of the rest of the line. The code below echos a; echo b instead of a b, how do I get it to run both commands?

import subprocess, shlex
def subprocess_cmd(command):
    process = subprocess.Popen(shlex.split(command), stdout=subprocess.PIPE)
    proc_stdout = process.communicate()[0].strip() 
    print proc_stdout

subprocess_cmd("echo a; echo b")
Paul
  • 5,756
  • 6
  • 48
  • 78
  • 1
    related: Here's [how to run multiple shell commands (and optionally capture their output) concurrently](http://stackoverflow.com/a/23616229/4279) – jfs Jul 26 '14 at 14:01

7 Answers7

118

You have to use shell=True in subprocess and no shlex.split:

import subprocess

command = "echo a; echo b"

ret = subprocess.run(command, capture_output=True, shell=True)

# before Python 3.7:
# ret = subprocess.run(command, stdout=subprocess.PIPE, shell=True)

print(ret.stdout.decode())

returns:

a
b
Nico Schlömer
  • 53,797
  • 27
  • 201
  • 249
bougui
  • 3,507
  • 4
  • 22
  • 27
  • 8
    Ah I see, I had `shell=True` tested but command.split() was breaking it (list). Also to note to other people reading, using `shell=True` is a security hazard, make sure you trust the input. – Paul Jul 19 '13 at 10:15
  • 3
    You cannot use `command.split()` with `shell=True`. Actually the argument of `subprocess.Popen` with `shell=True` must be a string and not a list. – bougui Jul 19 '13 at 10:19
  • 1
    I found this to be a bit simpler and didn't have the garbling of the output: def do_shell(self, command): self.proc=subprocess.Popen(command,shell=True) self.proc.wait() – Goblinhack May 13 '15 at 21:55
  • 2
    @Ravichandra: notice: `bash` in the title. On Windows, you probably want `echo a & echo b` command instead. – jfs Sep 22 '15 at 10:03
  • Your result runs `/bin/sh`, not `/bin/bash`. Add `executable='/bin/bash'` to the `Popen()` args to get a bash shell. – Earl Ruby Sep 10 '20 at 23:59
  • On windows this does not work, it returns "a; echo b" – Milan Oct 05 '22 at 15:17
  • @Milan, on windows, the default shell is batch, which does not honor ';' as a separator. – oldpride Jul 08 '23 at 23:57
31

I just stumbled on a situation where I needed to run a bunch of lines of bash code (not separated with semicolons) from within python. In this scenario the proposed solutions do not help. One approach would be to save a file and then run it with Popen, but it wasn't possible in my situation.

What I ended up doing is something like:

commands = '''
echo "a"
echo "b"
echo "c"
echo "d"
'''

process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = process.communicate(commands)
print out

So I first create the child bash process and after I tell it what to execute. This approach removes the limitations of passing the command directly to the Popen constructor.

admenva
  • 2,341
  • 19
  • 12
  • 5
    `subprocess.check_output(commands, shell=True)` works just fine. If there are bash-isms in commands then pass `executable='/bin/bash'`. – jfs Mar 26 '15 at 08:31
  • 10
    For Python 3 you might want to use: `out, err = process.communicate(commands.encode('utf-8'))` and `print(out.decode('utf-8'))` – Eyal Levin Jul 04 '16 at 08:04
  • If the commands are `;` separated you might also want to add `shell=True` to the `subprocess.Popen()` line. – ssanch Apr 29 '21 at 14:58
19

Join commands with "&&".

os.system('echo a > outputa.txt && echo b > outputb.txt')
FrancisWolcott
  • 287
  • 2
  • 4
4

If you're only running the commands in one shot then you can just use subprocess.check_output convenience function:

def subprocess_cmd(command):
    output = subprocess.check_output(command, shell=True)
    print output
Pierz
  • 7,064
  • 52
  • 59
1
>>> command = "echo a; echo b"
>>> shlex.split(command);
    ['echo', 'a; echo', 'b']

so, the problem is shlex module do not handle ";"

David.Zheng
  • 903
  • 8
  • 6
  • command.split() gives `['echo', 'a;', 'echo', 'b']` and also fails. ` – Paul Jul 19 '13 at 10:05
  • 1
    @bougui is right. Set "shell=True", the first argument of Popen need a command string like "echo a;echo b". and without "shell=True" the first argument of Popen should be a list, like:["echo","a"] – David.Zheng Jul 19 '13 at 10:18
1

Got errors like when I used capture_output=True

TypeError: __init__() got an unexpected keyword argument 'capture_output'

After made changes like as below and its works fine

import subprocess

command = '''ls'''

result = subprocess.run(command, stdout=subprocess.PIPE,shell=True)

print(result.stdout.splitlines())
Nanda Thota
  • 322
  • 3
  • 10
  • `capture_output` was only introduced in Python 3.7, which is quite old already, but apparently yours is older still. – tripleee Jun 17 '22 at 09:02
  • (The `shell=True` is useless here; `ls` runs fine without a shell, though nominally you should then pass the command as a list `["ls"]`.) – tripleee Jun 17 '22 at 09:03
0
import subprocess
cmd = "vsish -e ls /vmkModules/lsom/disks/  | cut -d '/' -f 1  | while read diskID  ; do echo $diskID; vsish -e cat /vmkModules/lsom/disks/$diskID/virstoStats | grep -iE 'Delete pending |trims currently queued' ;  echo '====================' ;done ;"


def subprocess_cmd(command):
    process = subprocess.Popen(command,stdout=subprocess.PIPE, shell=True)
    proc_stdout = process.communicate()[0].strip()
    for line in proc_stdout.decode().split('\n'):
        print (line)

subprocess_cmd(cmd)
neelam gour
  • 1