How can I write find and exec command in python

Question

I have a shell script that does

find /tmp/test/* -name "*.json" -exec python /python/path {} \;

it looks for all the JSON files in specific directories and executes the OTHER python script that I have..

How can I do this python scripting?

Answer 1

I'm not sure if I understood your question, if you are trying to execute a shell command from a python script, you can use os.system() :

import os
os.system('ls -l')

complete documentation

Answer 2

If you want to use Python instead of find, start with os.walk (official doc) to get the files. Once you have them (or as you them), act on them however you like.

From that page:

import os

for dirName, subdirList, fileList in os.walk(rootDir):
    print('Found directory: %s' % dirName)
    for fname in fileList:
        print('\t%s' % fname)
        # act on the file

Answer 3

import glob,subprocess
for json_file in glob.glob("/home/tmp/*.json"):
    subprocess.Popen(["python","/path/to/my.py",json_file],env=os.environ).communicate()

Answer 4

I guess you want to adjust your other python file /python/path/script.py, such that you only need this one file:

#!/usr/bin/env python
import sys
import glob
import os

#
# parse the command line arguemnts
#
for i, n in enumerate(sys.argv):
    # Debug output
    print "arg nr %02i: %s" % (i, n)

    # Store the args in variables
    # (0 is the filename of the script itself)
    if i==1:
        path = sys.argv[1] # the 1st arg is the path like "/tmp/test"
    if i==2:
        pattern = sys.argv[2] # a pattern to match for, like "'*.json'"

#
# merge path and pattern
# os.path makes it win / linux compatible ( / vs \ ...) and other stuff
#
fqps = os.path.join(path, pattern)

#
# Do something with your files
#
for filename in glob.glob(fqps):
    print filename

    # Do your stuff here with one file        
    with open(filename, 'r') as f:  # 'r'= only ready from file ('w' = write)
        lines = f.readlines()
    # at this point, the file is closed again!
    for line in lines:
        print line
        # and so on ...

Then you can use the one script like this /python/path/script.py /tmp/test/ '*.json'. (Without needing to write python in front, thanks to the very first line, called shebang. But you need to make it executable once, using chmod +x /python/path/script.py)

Of course you can omit the 2nd arg and assign a default value to pattern, or only use one arg in the first place. I did it this way to demonstrate os.path.join() and quoting of arguments that should not be extended by bash (compare the effect of using '*.json' and *.json on the printed list of arguments at the start)

Here are some information about better / more sophisticated ways of handling command line arguments.

And, as a bonus, using a main() function is advisable as well, to keep overview if your script gets larger or is being used by other python scripts.

Answer 5

What you need is

find /tmp/test/* -name "*.json" -exec sh -c "python /python/path {}" \;