get how much time python subprocess spends

Question

I'd like to time how long does the subprocess take. I tried to use

start = time.time()
subprocess.call('....')
elapsed = (time.time() - start)

However it's not very accurate (not sure related to multi-process or sth else) Is there a better way I can get how much time the subprocess really spends?

Thank you!

score 13 · Accepted Answer · answered May 22 '13 at 21:08

13

It depends on which time you want; elapsed time, user mode, system mode?

With resource.getrusage you can query the user mode and system mode time of the current process's children. This only works on UNIX platforms (like e.g. Linux, BSD and OS X):

import resource
info = resource.getrusage(resource.RUSAGE_CHILDREN)

On Windows you'll probably have to use ctypes to get equivalent information from the WIN32 API.

answered May 22 '13 at 21:08

Roland Smith

42,427
3
64
94

this answer looks fine for timing python code, but if we are running a subprocess, this may not be accurate, since Popen.wait() uses a busy loop. [https://docs.python.org/dev/library/subprocess.html#subprocess.Popen.wait] – ggg Nov 05 '14 at 01:17
@ggg Not sure if this is really a problem: Popen.wait() should run in the current process. Therefore, the resource usage of child processes should not be affected. – Steohan Mar 13 '17 at 15:52
1

[This other answer](https://stackoverflow.com/a/13933797/6196010) is quite more explicit about how to go about this. – M.M Nov 01 '19 at 00:14

score 7 · Answer 2 · answered May 22 '13 at 20:51

7

This is more accurate:

from timeit import timeit
print timeit(stmt = "subprocess.call('...')", setup = "import subprocess", number = 100)

answered May 22 '13 at 20:51

Rushy Panchal

16,979
16
61
94

score 3 · Answer 3 · answered Jul 24 '13 at 07:25

It took me a little while to implement Roland's solution due to the annoyingness of passing around python code as strings, so I thought I'd share a working example.

This script times an external program in the working directory, and redirects its standard output and standard error to files.

from timeit import timeit

reps = 500
stdout = open("add_numbers_outputs.log", 'w')
stderr = open("add_numbers_errors.log", 'w')
external_command = "./add_numbers"
parameter = str(1000000) # one million

call_arguments = """[
        '%s',
        '%s'], # pass additional parameters by adding elements to this list
        stdout=stdout,
        stderr=stderr
""" % (external_command, parameter)

print "Timing external command "+external_command+" with parameter "+parameter

time_taken = timeit(stmt = "subprocess.call(%s)" % call_arguments,
        setup = """import subprocess;
stdout = open("add_numbers_outputs.log", 'w');
stderr = open("add_numbers_errors.log", 'w')
""",
        number = reps) / reps

print "Average time taken for %s repetitions: %f seconds" % (reps, time_taken)

If it would make your code easier, you can pass a function to `timeit()` and avoid having to put any Python code inside of strings. — Brandon Rhodes, May 04 '16 at 19:10

get how much time python subprocess spends

3 Answers3

Linked