How to use timeit where each test requires random setup

Question

I have a function f(x) that takes as input a list x of 100 random floats between 0 and 1. Different lists will result in different running times of f.

I want to find out how long f takes to run on average, over a large number of different random lists. What's the best way to do this? Should I use timeit and if so is there a way I can do this without including the time it takes to generate each random list in each trial?

This is how I would do it without timeit (pseudocode):

for i = 1 to 10000:
    x = random list
    start = current time
    f(x)
    end = current time
    results.append(end - start)
return mean(results)

http://stackoverflow.com/questions/2245161/how-to-measure-execution-time-of-functions-automatically-in-python — sve, Oct 26 '13 at 11:37
@lnwvr lol, I never saw that. If I did, I'd just copy it, no need to make my own decorator! :P — Games Brainiac, Oct 26 '13 at 12:00

score 3 · Answer 1 · edited Nov 26 '13 at 13:22

You can make a timer decorator:

Here is some example code:

from time import time


class Timer(object):
    def __init__(self, func):
        """
        Decorator that times a function
        @param func: Function being decorated
        @type func: callable
        """
        self.func = func

    def __call__(self, *args, **kwargs):
        start = time()
        self.func(*args, **kwargs)
        end = time()
        return end - start


@Timer
def cheese():
    for var in xrange(9999999):
        continue

for var in xrange(100):
    print cheese()

Working example, with fewer loops.

score 1 · Answer 2 · answered Oct 26 '13 at 11:41

1

import timeit, random

def summer(myList):
    result = 0
    for num in myList:
        result += num
    return result

for i in range(10):
    x = [random.randint(0, 100) for i in range(100000)]
    print timeit.timeit("summer(x)", setup="from __main__ import x, summer", number = 100)

You can import the variable using from __main__ import x

answered Oct 26 '13 at 11:41

thefourtheye

233,700
52
457
497

Doesn't calling `timeit` a number of times in a loop defeat the purpose of using it though? – Flash Oct 26 '13 at 11:47
@Andrew You wanted to test the function with random inputs every time, right? Thats why its in a loop, with random input everytime. – thefourtheye Oct 26 '13 at 11:54
@Andrew Its the same, but with `timeit` way of doing it. – thefourtheye Oct 26 '13 at 11:57
1

`timeit` is not as flexible as I'd like. None the less, it is a way that you can achieve what you want. – Games Brainiac Oct 26 '13 at 12:03

score 0 · Answer 3 · 2013-10-27T21:24:30.603

I think this does the trick. It will execute the setup once per repeat and then execute stmt number=1 time. However I don't think this is that much better than the simple loop you posted.

import timeit

stmt = '[x*x*x for x in xrange(n)]'  # just an example 
setup = 'import random; n = random.randint(10, 100)'
r = 10000
times = timeit.repeat(stmt, setup, repeat=r, number=1)

print min(times), max(times), sum(times)/r

There is also a "cell mode" that you can use with timeit in the IPython shell, but it only returns the fasted time and there is no easy way to change that (?).

import random

%%timeit -r 10000 -n 1 n = random.randint(10,100)
var = [x*x*x for x in xrange(n)]

How to use timeit where each test requires random setup

3 Answers3