Popen piping 2 cmd hanging and not giving expected result

Question

I am trying to use grep and pipe it to uniq to get unique results...(greping for ip addresses here)

process = subprocess.Popen(['grep','-sRIEho', '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}', '/tmp/test'])
p2 = subprocess.Popen(['uniq'],stdin=process.stdout)    
stdout,stderr = p2.communicate()

The problem is that this hangs and doesn't show me unique results... Here's the '/tmp/test' file:

127.0.0.1
127.0.0.1
127.0.0.1
1.9.12.12
192.168.1.2
192.168.1.2
192.168.1.3
192.168.1.4

And the result well... It's the same Any idea what's happening here? Btw I cannot use Shell=True here (user supplied filename)

Answer 1

.communicate() returns None unless you pass PIPE i.e., both stdout, stderr are always None in your example. Also, you should close process.stdout in the parent so that grep would know if uniq dies prematurely. It doesn't explain why grep "hangs". The issue might be grep args such as -R (recursive, follows symlinks).

Try working solutions from How do I use subprocess.Popen to connect multiple processes by pipes? e.g.:

#!/usr/bin/env python3
from subprocess import Popen, PIPE

with Popen(['uniq'], stdin=PIPE, stdout=PIPE) as uniq, \
     Popen(['grep'] + grep_args, stdout=uniq.stdin):
    output = uniq.communicate()[0]

Or:

#!/usr/bin/env python
from plumbum.cmd import grep, uniq  # $ pip install plumbum

output_text = (grep[grep_args] | uniq)()