2

How to write the following line using subprocess library instead of commands. The idea is to get the same result but using subprocess.

commands.getoutput('tr -d "\'" < /tmp/file_1.txt > /tmp/file_2.txt')
Bach
  • 6,145
  • 7
  • 36
  • 61

3 Answers3

2

The equivalent command to commands.getoutput is subprocess.check_output:

from subprocess import check_output
out = check_output('tr -d "\'" < /tmp/file_1.txt > /tmp/file_2.txt', shell=True)
Ashwini Chaudhary
  • 244,495
  • 58
  • 464
  • 504
  • What would change if I used shell=False instead of shell=True? –  Feb 24 '14 at 13:30
  • See my answer below. By default you need to split your command. Whit shell=True you can write it as one string. See http://docs.python.org/2/library/subprocess.html#subprocess.check_output – Andrii Rusanov Feb 24 '14 at 13:32
  • @En_Py `shell=False` expects you to pass a list of arguments instead of string, and the redirection operators won't work with `shell=False`. Check the subprocess docs it is well documented. – Ashwini Chaudhary Feb 24 '14 at 13:38
  • I'm confused: I've read this and says that shell=True isn't safe to use: https://stackoverflow.com/questions/3172470/actual-meaning-of-shell-true-in-subprocess –  Feb 24 '14 at 14:01
  • @En_Py That's true, if input is coming from an unknown source then don't use `shell=True`. Check [this post](http://stackoverflow.com/questions/13332268/python-subprocess-command-with-pipe) for how to pipe or redirect when using shell=False. – Ashwini Chaudhary Feb 24 '14 at 14:04
  • Note: due to `> output_file`, it is pointless to use `check_output()` here. If you want to emulate `commands.getoutput()` more completely then you should capture stderr `{ cmd ; } 2>&1`. Here's an [implementation with `shell=False`](http://stackoverflow.com/a/21990959/4279) – jfs Feb 24 '14 at 14:50
1
import subprocess    

p=subprocess.Popen('tr -d "\'" < /tmp/file_1.txt > /tmp/file_2.txt',shell=True,stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output=p.communicate()[0]
print "o", output
Furquan Khan
  • 1,586
  • 1
  • 15
  • 30
  • 1
    +1 for emulating `{ cmd ; } 2>&1` (including `stderr`). Note: `commands.getoutput()` strips trailing newline. Look at [how `commands.getoutput()`it is implemented in terms of `subprocess` module](http://hg.python.org/cpython/file/a7a62a88380a/Lib/subprocess.py#l692) – jfs Feb 24 '14 at 15:04
0

You don't need shell in this case, to run tr command. And since you've redirected the child process' stdout; you also don't need subprocess.check_output():

from subprocess import check_call

with open("/tmp/file_1.txt", "rb") as input_file:
    with open("/tmp/file_2.txt", "wb") as output_file:
        check_call(["tr", "-d", "'"], stdin=input_file, stdout=output_file)

Note: unlike commands.getoutput() it doesn't capture stderr. If you want to get stderr of the subprocess as a separate string:

from subprocess import Popen, PIPE

with open("/tmp/file_1.txt", "rb") as input_file:
    with open("/tmp/file_2.txt", "wb") as output_file:
        p = Popen(["tr", "-d", "'"], stdin=input_file, stdout=output_file,
                  stderr=PIPE)
stderr_data = p.communicate()[1]

Also, you could replace tr call with a pure Python:

from functools import partial

chunksize = 1 << 15
with open("/tmp/file_1.txt", "rb") as input_file:
    with open("/tmp/file_2.txt", "wb") as output_file:
         for chunk in iter(partial(input_file.read, chunksize), b''):
             output_file.write(chunk.replace(b"'", b""))
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • But I can't figure out what chunksize = 1 << 15 function is. –  Feb 24 '14 at 15:21
  • @En_Py: `1 << 15` is `2**15` i.e., `32768`. Usually this values (buffer size) are powers of two. You could use `chunksize = io.DEFAULT_BUFFER_SIZE` for readability (changing `chunksize` may affect time performance). The code does the equivalent of `chunk = input_file.read(chunksize)` repeatedly until EOF. – jfs Feb 24 '14 at 15:37