Continuing script after failure on os.system call python

Question

I have a script I have written to scan a directory of text files and if it finds them, it creates a system call to run the script on the txt file. I am still working on a couple little bugs that causes some of my system calls to fail. I would like, however, for this NOT to kill my script. I would just like to be informed of the error then move on with my life. It seems any successful call returns 0 while any call that resulted in error returns n. I tried to compare the result with 0 but it never gets that far. Any suggestions on how I can accomplish this??

import sys, getopt, os

def main(argv):

    def scan_dir(path):
            temp_path = path
            print temp_path
            for file in os.listdir(path):
                    temp_path += file
                    if file.endswith(".txt"):
                            result = os.system("python callscript.py -i %s" % path)
                            if result != 0
                                    print "Error!"

                    temp_path = path


    def usage():
            print "usage:  dostuff.py [hi:]\n \
                  \t -h\t print usage\n \
                  \t -i\t directory path\n"
            sys.exit(2)

    if(len(argv) == 0):
            usage()

    path = ''

    try:
            opts, args = getopt.getopt(argv,"hi:",["path="])
    except getopt.GetoptError:
            usage()

    for opt, arg in opts:
            if opt == '-h':
                    usage()

            elif opt in ("-i", "--ipath"):
                    path = arg
    if path.endswith('/') == False:
            path += '/'

    scan_dir(path)



if __name__ == "__main__":
main(sys.argv[1:])

Why use `os` in the first place if your loop is in Python and the script is in Python? — physicalattraction, Jun 04 '15 at 16:47

Padraic Cunningham · Accepted Answer · 2015-06-04T17:27:08.617

You should be using the subprocess module in particular check_call, catching a CalledProcessError which would be raised for any non-zero exit status:

 from subprocess import check_call,CalledProcessError      

  try:
       check_call(["python", "callscript.py", "-i",path])
  except CalledProcessError as e:
       print e.message

Not that easy to follow your code, I would suggest not nesting all the other functions in main. I would also use glob to find the txt files:

from glob import  glob

def scan_dir(path):
    files = (os.path.join(path,f) for f in glob(os.path.join(path,"*.txt")))
    for fle in files:    
        try:
            check_call(["python", "callscript.py", "-i", fle])
        except CalledProcessError as e:
            print e.message

def usage():
    print "usage:  dostuff.py [hi:]\n \
          \t -h\t print usage\n \
          \t -i\t directory path\n"
    sys.exit(2)


if __name__ == "__main__":
    args = sys.argv[1:]
    if not args:
        usage()
    path = ''
    try:
        opts, args = getopt.getopt(args, "hi:",["path="])
    except getopt.GetoptError:
            usage()

    for opt, arg in opts:
        if opt == '-h':
                usage()
        elif opt in ("-i", "--ipath"):
                    path = arg
    if not path.endswith('/'):
            path += '/'
    scan_dir(path)

@kennedyl, no prob, you should really always log the errors or at least print them out instead of just ignoring them, the answer below suggests using Popen which will not wait for a return code and using shell=True is not usually a good idea. — Padraic Cunningham, Jun 04 '15 at 17:53
I got everything working. My apologies for the hard to read code earlier and thanks for your help. I tend to modularize my code once I have functionality. Below is my final product. I am curious, though, what the benefit of glob is over my implementation?? — kennedyl, Jun 04 '15 at 20:29
Thanks for this answer. I knew I had to do exception handling but didn't know what. Now I don't have to monitor my scripts and restart them. — user238607, Dec 15 '17 at 17:30

Damien Durand · Answer 2 · 2015-06-04T17:23:05.173

0

To catch errors and outputs without stopping your main process, I suggest using the subprocess module.

import subprocess

processToRun = "python callscript.py -i %s" % path
proc = subprocess.Popen (processToRun, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(proc.returncode)

edited Jun 04 '15 at 17:23

answered Jun 04 '15 at 16:52

Damien Durand

26
5

Not sure about the down vote here. check_call is only needed if you want an exception thrown. The main code running popen will keep going with its life even if the child process returns a non-zero code. See this other post: http://stackoverflow.com/questions/15316398/check-a-commands-return-code-when-subprocess-raises-a-calledprocesserror-except – Damien Durand Jun 04 '15 at 17:13

score 0 · Answer 3 · answered Jun 04 '15 at 16:56

If you are calling to invoke a python script, you should consider integrate that into your own python program.
Otherwise, you should use the subprocess module, this is recommended in the python documentation as shown in os.system. The documentation also suggests writing the replacement in this way substitute for os.system. Refer to section 17.1.4.3. for how to catch the exception.

score 0 · Answer 4 · answered Jun 04 '15 at 20:34

My current solution:

import sys, getopt, os    
from subprocess import check_call, CalledProcessError


def scan_dir(path):

    fail_count = 0
    success = []
    failure = []

    temp_path = path
    print temp_path
    for file in os.listdir(path):
            temp_path += file
            if file.endswith(".txt"):
                    try:
                            check_call(["python", "callscript.py","-i", temp_path])
                            success.append(file)
                    except CalledProcessError as e:
                            print e.message
                            failure.append(file)
                            fail_count += 1
                            continue
            temp_path = path

    if fail_count > 0:
            f = open("log.txt", "w")
            print ("%d Errors occurred during conversion. Check log.txt for details" % fail_count)
            f.write("The following files were successfully converted:\n")
            for s in success:
                    f.write("\t-%s\n" % s)

            f.write("\n\nThe following files failed to be converted:\n")
            for a in failure:
                    f.write("\t-%s\n" % a)
            f.close()
    else:
            print "All files converted successfully!"


def usage():
    print "usage:  dostuff.py [hi:]\n \
          \t -h\t print usage\n \
          \t -i\t directory path\n"
    sys.exit(2)

def main(argv):

    if(len(argv) == 0):
            usage()

    path = ''

    try:
            opts, args = getopt.getopt(argv,"hi:",["path="])
    except getopt.GetoptError:
            usage()

    for opt, arg in opts:
            if opt == '-h':
                    usage()
elif opt in ("-i", "--ifile"):
                    path = arg
    if path.endswith('/') == False:
            path += '/'

    scan_dir(path)


if __name__ == "__main__":
    main(sys.argv[1:])

Continuing script after failure on os.system call python

4 Answers4