python subprocess with pipe

Question

How to implement following shell script on python subprocess


grep -E "@|NM:" forward.sam | grep -v "XS:" | grep -v "^@" | cut -f1 > test.txt

I tried with following code; however, got the error.

command = ['grep']
command.append('-E')
command.append("'@|NM:'")
command.append("forward.sam")
command.append('|')
command.append('grep')
command.append('-v')
command.append("'XS:'")
command.append(">")
command.append("test.txt")

subprocess.check_call(command)

Error

grep: |: No such file or directory
grep: grep: No such file or directory
grep: 'XS:': No such file or directory
grep: >: No such file or directory
grep: test.txt: No such file or directory

Thanks for suggestions: @CharlesDuffy, @dpwrussell

Solution:

 p1 = subprocess.Popen(["grep", "-E", "@|NM:", samfile], stdout=subprocess.PIPE)
 p2 = subprocess.Popen(["grep", "-v", "XS:"], stdin=p1.stdout, stdout=subprocess.PIPE)
 p3 = subprocess.Popen(["grep", "-v", "^@"], stdin=p2.stdout, stdout=subprocess.PIPE)
 p4 = subprocess.Popen(["cut", "-f1"], stdin=p3.stdout, stdout=subprocess.PIPE)
 outfile = p4.communicate()[0] # outfile is bytes

`>` and `|` are part of the shell (e.g. bash) that you do not have when using subprocess. You can put the entire grep command inside a shell script or follow one of the several answers enumerated here: https://stackoverflow.com/questions/4965159/how-to-redirect-output-with-subprocess-in-python — dpwr, Sep 08 '20 at 15:31
...the _best_ practice is to have a separate `subprocess` object per pipeline component, as described at https://docs.python.org/3/library/subprocess.html#replacing-shell-pipeline... though here, it's hard to defend that you have a compelling reason to use external tools at all, when Python could do all the work internally. — Charles Duffy, Sep 08 '20 at 15:51

python subprocess with pipe

Solution:

0 Answers0