using call() to execute cat, with pipes and redirected output in python

Question

I've this shell command:

cat input | python 1.py > outfile

input is a text file with values

and 1.py is:

t = int(raw_input())

while t:
    n = int(raw_input())
    print n
    t -= 1

It runs perfectly when I enter it in the terminal.

However, when I run this from Python using this code:

from subprocess import call
script = "cat input | python 1.py > outfile".split()
call(script)

I get:

3
1
4
5

cat: |: No such file or directory
cat: python: No such file or directory
t = int(raw_input())

while t:
    n = int(raw_input())
    print n
    t -= 1
cat: >: No such file or directory
cat: outfile: No such file or directory

cat: |: No such file or directory
cat: python: No such file or directory
cat: >: No such file or directory
cat: outfile: No such file or directory

How do I get it right?

score 6 · Accepted Answer · answered May 27 '14 at 11:57

By default, the arguments to call are executed directly, not passed to a shell, so the pipelines and IO redirections are not processed. Use shell=True and a single string argument instead.

from subprocess import call
script = "cat input | python 1.py > outfile"
call(script, shell=True)

However, it's better to let Python handle the redirections itself without involving the shell. (Note that cat here is unnecessary; you could use python 1.py < input > outfile instead.)

from subprocess import call
script="python 1.py".split()
with open("input", "r") as input:
    with open("outfile", "w") as output:
        call(script, stdin=input, stdout=output)

what if I have parantheses in my command like below: 'cat <(cat bal_fold_2.csv) <(tail -n+2 bal_fold_3.csv) > bal_train.csv' — Jumabek Alikhanov, Aug 23 '20 at 10:08

score 2 · Answer 2 · edited May 23 '17 at 12:32

2

Unlike in a shell where | and > are considered as special redirection symbols, Python's call() sees them as normal and are passed as well as normal arguments to cat. You may consider calling a command via a shell with other methods like os.system:

import os
os.system("cat input | python 1.py > outfile")

Answer forwarded from Running a command completely indepenently from script.

edited May 23 '17 at 12:32

Community

1
1

answered May 27 '14 at 11:57

konsolebox

72,135
12
99
105

The Python documentation itself prefers the `subprocess` module to using `os.system`. – chepner May 27 '14 at 12:56

using call() to execute cat, with pipes and redirected output in python

2 Answers2