Python - subprocess with Quotes and Pipe Grep

Question

im having an issue trying to get a simple grep command into python. I want to take the output of the following command in a file or a list.

grep -c 'some thing' /home/user/* | grep -v :0

This is what I have, but its not working at all...

thing = str(subprocess.Popen(['grep', '-c', 'some thing', '/home/user/*', '|', 'grep', '-v', ':0'], stdout=subprocess.PIPE)

Basically I need to search files in a directory and return a result if my string is missing from any of the files in the directory.

Working Code (Thanks!!):

thing = subprocess.Popen(('grep -c "some thing" /home/user/* | grep -v ":0"' ),shell=True, stdout=subprocess.PIPE)

Python 3.7+: `thing = subprocess.run(cmd, shell=True, capture_output=True, universal_newlines=True)` — Ali Abbasinasab, Sep 18 '19 at 22:18

score 5 · Accepted Answer · answered Jan 12 '15 at 22:34

5

The pipe | is a shell feature. You have to use Popen with shell=True to use it.

answered Jan 12 '15 at 22:34

Aran-Fey

39,665
11
104
149

score 2 · Answer 2 · edited May 23 '17 at 12:23

To emulate the shell pipeline in Python, see How do I use subprocess.Popen to connect multiple processes by pipes?:

#!/usr/bin/env python
import os
from glob import glob
from subprocess import Popen, PIPE

p1 = Popen(["grep", "-c",  'some thing'] + glob(os.path.expanduser('~/*')),
           stdout=PIPE)
p2 = Popen(["grep", "-v", ":0"], stdin=p1.stdout)
p1.stdout.close()
p2.wait()
p1.wait()

To get output as a string, set stdout=PIPE and call output = p2.communicate()[0] instead of p2.wait().

To suppress error messages such as "grep: /home/user/dir: Is a directory", you could set stderr=DEVNULL.

You could implement the pipeline in pure Python:

import os
from glob import glob

for name in glob(os.path.expanduser('~/*')):
    try:
        count = sum(1 for line in open(name, 'rb') if b'some thing' in line)
    except IOError:
        pass # ignore
    else:
        if count: # don't print zero counts
           print("%s:%d" % (name, count))

Python - subprocess with Quotes and Pipe Grep

2 Answers2

Linked