subprocess not running the command generated though the command works on terminal

Question

cmd= ["sudo", "cat", "{filepath}".format(filepath=filepath), "|","egrep", "-v","\'{filteruser}\'".format(filteruser=filteruser)]

fileformat and filteruser can be blank too

config file below

[plugin_haproxy]
user= tara
host=localhost
filepath=/etc/haproxy/haproxy.global.inc
filter=

This is the command I want to run on subprocess, checking the above variable values with pdb shows bellow value and looks great

['sudo', 'cat', '/etc/haproxy/haproxy.global.inc', '|', 'egrep', '-v', "''"]

Manullay running the code sudo cat /etc/haproxy/haproxy.global.inc | egrep -v " '' " on terminal works great

Why is subprocess not able to process it.

"cat: |: No such file or directory\ncat: egrep: No such file or directory\ncat:

try just this `['sudo', 'cat', '/etc/haproxy/haproxy.global.inc']` and see if it works — alpheus, Mar 29 '17 at 05:30
no I need a filter too egrep might be available so am using that , but thanks that helped me to figure out the problem is after that — Tara Prasad Gurung, Mar 29 '17 at 05:33
I am just trying to get to the root of the problem, just check and see if only cat command works and we can take it from there — alpheus, Mar 29 '17 at 05:34
try this `subprocess.call(["sudo cat /etc/haproxy/haproxy.global.inc | egrep -v ''"], shell=True)` — alpheus, Mar 29 '17 at 05:40
`p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)` but didn't help — Tara Prasad Gurung, Mar 29 '17 at 05:43
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/139318/discussion-between-alpheus-and-tara-prasad-gurung). — alpheus, Mar 29 '17 at 05:44

Ali Tajeldin · Answer 1 · 2017-03-29T07:18:14.857

3

your shell will take a command such $ a | b | c and turn it into three separate processes (a,b,c) and attach the output of one to the input of the next in the chain.

['sudo', 'cat', '/etc/haproxy/haproxy.global.inc', '|', 'egrep', '-v', "''"]

The above will execute the sudo command (which will in-turn fork the cat command and pass all the remaining args to cat). It doesn't understand the "|" symbol. It just treats it as another arg. It is cat that is complaining that it can not open "|", "egrep", etc at it is treating these as file names to cat.

You can try the Popen with shell=True. I have not tried that and not sure if it will handle pipes.

The other option is to use Popen to execute the sudo command only (no filter) and then use Popen.communicate to read the output from the command in python and do the empty line filtering in python.

EDIT: I did a quick check for the shell=True. Here is the script below:

#!/usr/bin/env python
import subprocess

subprocess.call(["ls", "-l", "|", "cat"], shell=False)

With shell=False, I get the following error: ls: cat: No such file or directory and ls: |: No such file or directory. This is expected as ls is trying to list the contents of | and cat.
with shell=True, I get the desired output. That is, the output of ls is piped to cat. The shell is processing the |.

edited Mar 29 '17 at 07:18

answered Mar 29 '17 at 06:20

Ali Tajeldin

61
4

shell=True is really not going to help – Tara Prasad Gurung Mar 29 '17 at 07:01
Why not? You're trying to use shell pipes, this should make it work. – cha0site Mar 29 '17 at 08:49
`p = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)` This is how I have passed shell=True and as mentioned before not working. filtering from python might work but that dont want extra burden and wont fulfill my need i want the filter with egrep not by parsing – Tara Prasad Gurung Mar 29 '17 at 09:05
`command = ["ssh", "{user}@{host}".format(user=user, host=host), "sudo", "cat", "/etc/passwd", "|", "egrep", "-v", "\"{filters}\"".format(filters=filteruser)]` This is another module of mine where I have done the filtering too but only the file path is directly provided. Why providing the path too dynamically not working. 2 steps might solve but why is this working than where I have a filter too in single process – Tara Prasad Gurung Mar 29 '17 at 09:15

score 0 · Answer 2 · answered Mar 29 '17 at 08:52

0

You can either use shell=True as was already suggested, or use subprocess to pipe:

p1 = subprocess.Popen(['sudo', 'cat', '/etc/haproxy/haproxy.global.inc'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['egrep', '-v', "''"], stdin=p1.stdout,stdout=subprocess.PIPE)

answered Mar 29 '17 at 08:52

cha0site

10,517
3
33
51

`command = ["ssh", "{user}@{host}".format(user=user, host=host), "sudo", "cat", "/etc/passwd", "|", "egrep", "-v", "\"{filters}\"".format(filters=filteruser)]` This is another module of mine where I have done the filtering too but only the file path is directly provided. Why providing the path too dynamically not working. 2 steps might solve but why is this working than . – Tara Prasad Gurung Mar 29 '17 at 09:13

score 0 · Answer 3 · edited May 23 '17 at 12:10

Command and filter above

command = ["sudo", "cat", "{filepath}".format(filepath=filepath)]         
userfilter = ["egrep", "-v", "\"{filteruser}\"".format(filteruser=filteruser)***]

Subprocess

cmdout = subprocess.Popen(command, stdout=subprocess.PIPE)
filtration = subprocess.Popen(runfilter, stdin=cmdout.stdout,stdout=subprocess.PIPE)

output, err = filtration.communicate()

Yes,I used a two steps solutions could not find a single best refer here,Python subprocess command with pipe

subprocess not running the command generated though the command works on terminal

config file below

3 Answers3