Running a Bash command on files with Python

Question

So I'm creating a script that needs to go through all the files on a server and run each of those files names followed by the command "ll" and then take the output of that command and print it to a txt file.

example:

folder/filename.txt ll

output: SoMETHINGSomethingSomethingother - This is sent to a output.txt file

folder/subfolder/filename3.txt ll

output: SoMETHINGSomethingSomethingother - This is sent to a output.txt file

This is what I have so far:

import os

with open("output.txt", "w") as a:
    for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
       for filename in files:
         f = os.path.join(filename)
         m = f + ' ll'

         a.write(str(m) + os.linesep)

So what I'm trying to figure out now is how to make the file names that are printed out run with the "ll" command. So far this code will write the names of all the files in that folder and its subfolders in to my output.txt file.

Does anybody have any ideas?

check `subprocess` module http://docs.python.org/library/subprocess.html — avasal, Aug 31 '12 at 08:33
possible duplicate of [Running shell command from python and capturing the output](http://stackoverflow.com/questions/4760215/running-shell-command-from-python-and-capturing-the-output) — Shawn Chin, Aug 31 '12 at 08:34
It's worth noting that in Perl you can write to a pipeline by opening the file (eg `open(my $pipeline, "| command | command > file");`), but Python doesn't allow this. — Eliot Ball, Aug 31 '12 at 08:45
@Eliot. Python performs that function. It would be odd if Python used the same syntax as Perl don't you think? — Paddy3118, Aug 31 '12 at 10:46
@Paddy3118 yes, I'm just pointing out for general knowledge :) — Eliot Ball, Aug 31 '12 at 11:08

score 1 · Answer 1 · edited May 23 '17 at 12:12

1

Use os.system():

import os

with open("output.txt", "w") as a:
    for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
        for filename in files:
            f = os.path.join(filename)
            m = f + ' ll > output.txt'

            os.system(m)

This will send only the standard output to the output.txt file. If you want to send error messages to output.txt as well, use m = f + ' ll > output.txt 2>&1' instead.

Explanation: os.system(command_string) will execute the command command_string in your system as if you typed that command into a terminal. The > operator is standard in Windows and Linux to redirect standard output from a command into a file. The 2>&1 extra argument at the end is the only not-so-clear part: it redirects standard error to the same place where standard output is going. See more about this last part here.

edited May 23 '17 at 12:12

Community

1
1

answered Aug 31 '12 at 08:37

HerrKaputt

2,594
18
17

Well it turns out I can't test this out until Thursday but I want to first say thank you for answering. I believe this will work. Also thank you for explaining it its really helped. Once I check on Thursday I will let you know if it worked. – Adilicious Aug 31 '12 at 09:15
We should replace `os.system` calls by their `subprocess` counterparts. Therefore a downvote. – Dr. Jan-Philip Gehrcke Aug 31 '12 at 09:19
@user1636295: `subprocess` is more general than `os.system`. However, `os.system` is not being deprecated, so there's no big problem in using it, even though the Python docs suggest [1] that people use `subprocess`. You can replace the `os.system(m)` line with `subprocess.call(m,shell=True)` [1]. I prefer `system` for these cases because it is less verbose. [1]http://docs.python.org/library/subprocess.html – HerrKaputt Aug 31 '12 at 10:23

consumer · Answer 2 · 2012-08-31T08:41:52.050

In order to run the files with the "ll" command you can use subprocess module available in python.

Your revised code will be:-

import os
import subprocess
import shlex

with open("output.txt", "w") as a:
    for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
       for filename in files:
         f = os.path.join(filename)
         m = f + ' ll'

         cmd_args = shlex.split(m)
         output = subprocess.check_output(cmd_args)
         a.write(output + os.linesep)

Running a Bash command on files with Python

2 Answers2