4

There are many posts here on SO, like this one: Store output of subprocess.Popen call in a string

There is problem with complicated commands. For example, if I need to get output from this

ps -ef|grep something|wc -l

Subprocess won't do the job, because argument for subprocess is [name of program, arguments], so it is not possible to use more sophisicated commands (more programs, pipes, etc.).

Is there way to capture the output of a chain of multiple commands?

Community
  • 1
  • 1
3p1i4
  • 51
  • 1
  • 5

3 Answers3

5

Just pass the shell=True option to subprocess

import subprocess
subprocess.check_output('ps -ef | grep something | wc -l', shell=True)
John La Rooy
  • 295,403
  • 53
  • 369
  • 502
5

For a no-shell, clean version using the subprocess module, you can use the following example (from the documentation):

output = `dmesg | grep hda`

becomes

p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

The Python program essentially does here what the shell does: it sends the output of each command to the next one in turn. An advantage of this approach is that the programmer has full control on the individual standard error outputs of the commands (they can be suppressed if needed, logged, etc.).

That said, I generally prefer to use instead the subprocess.check_output('ps -ef | grep something | wc -l', shell=True) shell-delegation approach suggested by nneonneo: it is general, very legible and convenient.

Eric O. Lebigot
  • 91,433
  • 48
  • 218
  • 260
3

Well, another alternative would just be to implement part of the command in plain Python. For example,

count = 0
for line in subprocess.check_output(['ps', '-ef']).split('\n'):
    if something in line: # or re.search(something, line) to use regex
        count += 1
print count
nneonneo
  • 171,345
  • 36
  • 312
  • 383
  • May I suggest `count = sum(something in line for line in subprocess.check(…))`? This is more direct. – Eric O. Lebigot Feb 20 '13 at 02:41
  • While I like generators, I'm also generally averse to having really long lines in code. It's just a stylistic thing; your approach is just as good. – nneonneo Feb 20 '13 at 02:43
  • 1
    The ps command is just an example. Sure, I can emulate trivial grep functionality and functionality of wc in python, but there are other combinations of commands. Saving output and then doing pipe "manually" (new subprocess call based on output from previous is possibility, but it is not elegant) – 3p1i4 Feb 20 '13 at 02:47
  • While I understand that long lines can be a pain, I think that the structure here is simple enough to not be really confusing. That said, I find that the choice between both approaches goes beyond style: with a loop, one has to read the whole code in order to understand what it does (basically an incremental count of the matching lines), whereas with the `count = sum(…)`, one sees immediately that `count` is set to the number of matching lines. This is why list comprehensions are usually more legible than a `result = []; for (…): result.append()` loop. – Eric O. Lebigot Feb 20 '13 at 02:49
  • 1
    In reality you'd probably store `subprocess.check(…)` in a variable and use a shorter name for `something`, so the generator expression will become clearer – John La Rooy Feb 20 '13 at 02:55