Setting a timeout during a loop in Python?

Question

So I'm running a loop through a directory where an external script, run by subprocess.Popen, is moving through the directory and performing calculations on each file. The external script is a bit unstable and occasionally freezes when it runs into files that it doesn't know how to handle. Is there a way to add a timeout function to subprocess.Popen so that I can skip over that file and move on to the next?

EDIT: Here's my loop:

def automate():
    os.chdir("/home/mlts/dir")
    working_dir = b"/home/mts/dir"
    for filename in os.listdir(working_dir):
        if filename.endswith(".MTS"):
            try:
                print("Performing calculations on {filename!r}...".format(**vars()))
                try:
                    os.remove("mts_tbl.txt")
                except OSError:
                    pass
                time.sleep(3)
                p = Popen(["run_command.command", "-f", "a"], cwd=working_dir, stdin=PIPE)
                p.communicate(input=b"\n".join([b"1", str(filename), b"8", b"alloy-liquid", b"0", b"x", b"5", b"4", b"-1.6", b"4", b"1", b"0"]))

One can help you over timeout part as is but for skip over the file script freezes on, you must provide code — MohitC, Oct 17 '15 at 22:55
To do 2 things at once, 1=run the function, 2=cancel after a certain amount of time has passed, you use Multiprocessing or Threading. Personally, I would call the function as a multiprocessing process, and terminate the process after the allowed time has passed. Doug Hellmann's page is at https://pymotw.com/2/multiprocessing/basics.html#terminating-processes — , Oct 17 '15 at 23:53
Would this approach allow for the continuation of the loop onto the next file in the directory? — Scott, Oct 18 '15 at 04:39
unrelated: don't call `str(filename)`; it is pointless on Python 2 (it is already has `str` type there) and it fails on Python 3; pass `filename` as is as shown in [my answer](http://stackoverflow.com/a/33170992/4279) — jfs, Oct 18 '15 at 12:01
pass `timeout` parameter to `.communicate()`, see [subprocess with timeout](http://stackoverflow.com/a/12698328/4279) — jfs, Oct 18 '15 at 12:02
is it intentional that `os.chdir()` argument differs from `working_dir`? — jfs, Oct 18 '15 at 12:05
I'm pretty new to coding in general and I probably have a lot of redundant lines. Only thing that matters to me right now is that it works (and it does, minus the timeout issue). I'll clean it up later. — Scott, Oct 18 '15 at 14:06

score 0 · Accepted Answer · answered Oct 18 '15 at 12:46

To force timeout on Python 2 (where .communicate(timeout=) doesn't exist in stdlib), you could use threading.Timer():

p = Popen(...)
t = Timer(3, p.kill) # kill in 3 seconds
t.start()
p.communicate(...)
t.cancel() # no need to kill, the process is dead already

The complete example:

import os
import traceback
from glob import glob
from subprocess import Popen, PIPE
from threading import Timer

os.chdir("/home/mls/dir") #XXX assume mlts is a typo in the question
for filename in glob(b"*.MTS"):
    print("Performing calculations on {filename!r}...".format(**vars()))
    try:
        os.remove("mts_tbl.txt")
    except OSError:
        pass # ignore
    try:
        p = Popen(["run_command.command", "-f", "a"], stdin=PIPE) #XXX no cwd
        t = Timer(3, p.kill)
        t.start()
        p.communicate(input=b"\n".join([b"1", filename,
                b"8\nalloy-liquid\n0\nx\n5\n4\n-1.6\n4\n1\n0"]))
        t.cancel()
    except Exception:
        traceback.print_exc()

This code assumes that you want to run the child process in the current working directory of the parent Python script.

On Python 3 or if subprocess32 is installed; you could pass timeout parameter to .communicate() method instead of using Timer().

Setting a timeout during a loop in Python?

1 Answers1