Execute df | grep -w "/" not parsing output correctly

Question

I am trying to run the shell command df -h | grep -w "/" using python to watch the root partition usage and wanted to avoid shell=True option for security.

The code I tried as follows:

import subprocess
p1 = subprocess.Popen(['df', '-h'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['grep', '-w', '"/"'], stdin=p1.stdout, stdout=subprocess.PIPE)
output=p2.communicate()[0]
print(output)

The output I get is:

$ ./subprocess_df_check.py 
b''

Expected output is:

$ df -h | grep -w "/"
/dev/sdd        251G  4.9G  234G   3% /

I think grep may be looking for the literal pattern `"/"` including the quotes. Try taking out the double quotes. You don't need them anyway, since `/` is not a shell special character. — John Gordon, Dec 18 '21 at 19:05
Excellent, that was the issue. But the output now printed as `(b'/dev/sdd 251G 4.9G 234G 3% /\n', None)`. How do you remove those additional "b" "\n", "None" stuff from the output?. — vjwilson, Dec 18 '21 at 19:09
`'grep', '-w', '/'` -- the double quotes are for the shell, not for `df`. When you have no shell, you need no shell syntax. — Charles Duffy, Dec 18 '21 at 20:03
@JohnW, if you still have the parenthesis and a `None`, that means your real code didn't have the `[0]`, so you were printing a `(stdout, stderr)` tuple instead of just the stdout. Make your code match what you showed in the question and you won't have that problem. (As for `b''`, we have hundreds of already-answered questions about that in the knowledgebase: It means you have a bytestring instead of a Unicode string). — Charles Duffy, Dec 18 '21 at 20:06

Andreas Florath · Answer 1 · 2021-12-18T20:09:19.267

2

Don't use subprocess with df and / or grep. If you already use python, you can use the statvfs function call like:

import os
import time

path = "/"

while True:
    info = os.statvfs(path)
    print("Block size [%d]  Free blocks [%d]  Free inodes [%d]"
          % (info.f_bsize, info.f_bfree, info.f_ffree))
    time.sleep(15)

edited Dec 18 '21 at 20:09

answered Dec 18 '21 at 19:29

Andreas Florath

4,418
22
32

score 2 · Answer 2 · answered Dec 18 '21 at 20:04

The immediate problem is the unnecessary quotes being added.

p2 = subprocess.Popen(['grep', '-w', '"/"'], stdin=p1.stdout, stdout=subprocess.PIPE)

is not equivalent to the shell command grep -w "/". Instead, it's equivalent to the shell command grep -w '"/"', (or grep -w \"/\", or any other means of writing an argument vector that passes literal double-quote characters on the last non-NUL element of grep's argument vector) and wrong for the same reasons.

Use '/', not '"/"'.

tripleee · Answer 3 · 2021-12-19T12:51:17.683

1

Running grep in a separate subprocess is certainly unnecessary. If you are using Python, you already have an excellent tool for looking for substrings within strings.

df = subprocess.run(['df', '-h'],
    capture_output=True, text=True, check=True)
for line in df.stdout.split('\n')[1:]:
    if '/' in line:
        print(line)

Notice also how you basically always want to prefer subprocess.run over Popen when you can, and how you want text=True to get text rather than bytes. Usually you also want check=True to ensure that the subprocess completed successfully.

edited Dec 19 '21 at 12:51

answered Dec 19 '21 at 12:43

tripleee

175,061
34
275
318

1

You're most welcome. For (much) more on this topic, see also https://stackoverflow.com/questions/4256107/running-bash-commands-in-python/51950538#51950538 – tripleee Dec 19 '21 at 12:48

vjwilson · Accepted Answer · 2021-12-19T12:38:12.673

Ok figured out the whole thing.

import subprocess
p1 = subprocess.Popen(['df', '-h'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['grep', '-w', '/'], stdin=p1.stdout, stdout=subprocess.PIPE)
output=p2.communicate()[0].split()[4]
print("Root partition is of", output.decode(), "usage now")

Removed unnecessary double quotes, changed from subprocess.Popen(['grep', '-w', '"/"'] to subprocess.Popen(['grep', '-w', '/']. The double quotes are for the shell, not for df. When you have no shell, you need no shell syntax.
On output=p2.communicate()[0].split()[4], the [0] picks only stdout, not the stderr, which is None if no error. Then split()[4] cuts the 4th column which is disk usage percent value from shell df command.
output.decode(), the decode() is to decode the encoded bytes string format and avoid outputting character b in front of the result. Refer here

So the output of the script is:

$ ./subprocess_df_check.py
Root partition is of 3% usage now

Execute df | grep -w "/" not parsing output correctly

4 Answers4