Using && in subprocess.Popen for command chaining?

Question

I'm using subprocess.Popen with Python, and I haven't come across an elegant solution for joining commands (i.e. foobar&& bizbang) via Popen.

I could do this:

p1 = subprocess.Popen(["mmls", "WinXP.E01"], stdout=subprocess.PIPE)
result = p1.communicate()[0].split("\n")
for line in result:
    script_log.write(line)

script_log.write("\n")

p1 = subprocess.Popen(["stat", "WinXP.E01"], stdout=subprocess.PIPE)
result = p1.communicate()[0].split("\n")
for line in result:
    script_log.write(line)

But that really isn't very aesthetically pleasing (especially if I'm daisy-chaining multiple commands via Popen.

I'd like to replicate this output in as few command blocks as possible.

not@work ~/ESI/lab3/images $ mmls WinXP.E01 && echo -e "\n" && stat WinXP.E01
DOS Partition Table
Offset Sector: 0
Units are in 512-byte sectors

     Slot    Start        End          Length       Description
00:  Meta    0000000000   0000000000   0000000001   Primary Table (#0)
01:  -----   0000000000   0000000062   0000000063   Unallocated
02:  00:00   0000000063   0020948759   0020948697   NTFS (0x07)
03:  -----   0020948760   0020971519   0000022760   Unallocated


  File: `WinXP.E01'
  Size: 4665518381  Blocks: 9112368    IO Block: 4096   regular file
Device: 14h/20d Inode: 4195953     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    nott)   Gid: ( 1000/    nott)
Access: 2013-03-16 23:20:41.901326579 -0400
Modify: 2013-03-04 10:05:50.000000000 -0500
Change: 2013-03-13 00:25:33.254684050 -0400
 Birth: -

Any suggestions?

Note: I'd like to avoid typing this into subprocess.Popen

p1 = subprocess.Popen(["mmls WinXP.E01 && echo -e '\n' && stat WinXP.E01"], stdout=subprocess.PIPE)

When you say "chaining" you just mean running the processes in sequence, right? Not passing the output of one program to the input of the next, like you can do with pipes? If so, it should be easy to come up with a function that takes a sequence of programs to run and returns a string (or list of lines) with the combined output. Just make a loop! — Blckknght, Mar 18 '13 at 00:46

score 9 · Answer 1 · edited Feb 01 '23 at 10:40

9

&& is a shell operator, Popen does not use a shell by default.

If you want to use shell functionality use shell=True in your Popen call, but be aware that it is slightly slower/more memory intensive.

p1 = subprocess.Popen(["mmls", "WinXP.E01", "&&", "echo", "-e", "\"\n\"", "&&", "stat", "WinXP.E01"],
                      stdout=subprocess.PIPE, shell=True)

edited Feb 01 '23 at 10:40

vvvvv

25,404
19
49
81

answered Mar 18 '13 at 01:19

Matt

432
2
8

What would be a more favourable solution that isn't slow and/or memory intensive? – Mar 18 '13 at 01:38
1

Doing two separate commands as you give in your example, or placing them in a loop. – Matt Mar 19 '13 at 00:14
2

Warning! using a shell is also a significant security risk. Avoid if possible! [link](https://stackoverflow.com/questions/3172470/actual-meaning-of-shell-true-in-subprocess) – Connor May 19 '19 at 02:30
How can it be avoided in this case? I feel the question is basically unanswered, but I don't know the answer. – Jonathan Rayner Apr 12 '22 at 01:08

score 2 · Answer 2 · answered Mar 18 '13 at 02:38

How about this:

from subprocess import Popen, PIPE

def log_command_outputs(commands):
    processes = [Popen(cmd, stdout=PIPE) for cmd in commands]
    outputs = [proc.communicate()[0].split() for proc in processes]
    for output in outputs:
        for line in output:
            script_log.write(line)
        script_long.write("\n")

This starts the commands in parallel, which might make it a little faster than doing them one by one (but probably not by a large margin). Since the communicate calls are sequential though, any command that has a large output (more than a pipe's buffer) will block until it's turn comes to be cleaned up.

For your example command chain, you'd call:

log_command_outputs([["mmls", "WinXP.E01"], ["stat", "WinXP.E01"]])

'&&' shell operator is used to call the next command only if previous is succeeded. Usually this is done because the next command depends on the output of the previous one. Running the commands in parallel won't work as expected in this case. — Alex Che, Jul 09 '20 at 15:29

Using && in subprocess.Popen for command chaining?

Any suggestions?

2 Answers2

Linked

Related