Is there a better way to do run time analysis than this?

Question

I currently have 2 different functions with options to vectorise them: acc_rej_sine(max_iter, algorithm=None) and analytical_sine(max_iter, algorithm=None) for which I'm trying to compare their run time against the number of iterations computed. i.e. compare all 4 methods; 2 looped, 2 vectorised. Essentially my code goes something like this:

def analytical_sine(max_iter, algorithm=None):
    if algorithm is None:
        count = 0
        analytical_hist = []
        for i in range(max_iter):
            count += 1
            progress = round((count/max_iter)*100)
            sys.stdout.write('\r' + str(progress) + '%')
            uni_dist = np.random.uniform(0, 1)
            arccos = np.arccos(1 - 2*uni_dist)
            analytical_hist.append(arccos)
    elif algorithm is "vectorise":
        analytical_hist = np.arccos(1 - 2*np.random.uniform(0, 1, max_iter))

    return analytical_hist

def acc_rej_sine(max_iter, algorithm=None):
    x = np.random.uniform(0, np.pi, max_iter)
    y = np.random.rand(max_iter)
    if algorithm is None:
        accepted_x = []
        j = count = 0
        for i in range(max_iter):
            count += 1
            progress = round((count/max_iter)*100)
            sys.stdout.write('\r' + str(progress) + '%')
            if y[i] <= np.sin(x[i]):
                accepted_x.append(x[i])
                j += 1
    elif algorithm is "vectorise":
        accepted_x = np.extract((y <= np.sin(x)), x)

    return accepted_x

def runtime(func, runs, max_iter, algorithm=None):
    time = []
    for i in range(runs):
        start = timer()
        func()
        end = timer()
        time.append((end-start))
    error = np.std(time)
    time = sum(time)/runs

    return time, error

def time_analysis():
    time1, time2, time3, time4 = [], [], [], []
    error1, error2, error3, error4 = [], [], [], []
    for i in np.arange(1, 8, 1):
        max_iter = 10**i
        time, error = runtime(analytical_sine, 5, int(max_iter))
        time1.append(time)
        error1.append(error)
        time, error = runtime(analytical_sine, 5, int(max_iter), "vectorise")
        time2.append(time)
        error2.append(error)
        time, error = runtime(acc_rej_sine, 5, int(max_iter))
        time3.append(time)
        error3.append(error)
        time, error = runtime(acc_rej_sine(max_iter), 5, int(max_iter), "vectorise")
        time4.append(time)
        error4.append(error)
    return [time1, time2, time3, time4], [error1, error2, error3, error4]

# to run the code I would probably do something like this
time, error = time_analysis()
#then if I wanna plot number of iterations vs run time with errors I would probably do something along the lines of
plt.plot(max_iter, time) # max_iter would be a list of [10**i for i in np.arange(1, 8, 1)]
plt.errorbar(error)

So the idea is my runtime() function would allow me to put in any of the 4 functions that I want to compare(which currently still isn't working yet and I can't/haven't figured out why), and run it for 5 times, work out the mean run time and standard deviation as the error. In return my time_analysis() function would run runtime() for different functions for different max_iter(max iterations) which goes like [10, 1E2, 1E3, 1E4, 1E5, 1E6, 1E7] so I can plot max iterations against run time. However, this whole method seems quite cumbersome and inelegant, as my time_analysis() requires me to repeatedly work out time and error and append it to a list. Is there a better way of timing this?(Also my runtime() doesn't work yet lol because my algorithm argument seems to be making something not callable)

You should just use the built-in `timeit` library, if only for the sake of not re-inventing the wheel, but it also handles all sorts of things which you are ignoring in your own (gc, making sure you are using the correct timer for your system, etc). As an aside, please always use the generic [python] tag for all python questions — juanpa.arrivillaga, Apr 04 '19 at 19:09
yea but how can I use `timeit` in this case to time my function **and** get the output of the functions at the same time so I can plot it? — user3613025, Apr 04 '19 at 19:25
BTW: `elif algorithm is "vectorise"` is a bug. *Don't use `is` to test for equality*, `is` tests for identity. — juanpa.arrivillaga, Apr 04 '19 at 19:28
Why do you need to get the output? [Here's](https://stackoverflow.com/questions/8220801/how-to-use-timeit-module) a basic question on the module. [Here are the docs](https://docs.python.org/3/library/timeit.html). What exactly are you having trouble with? Can you add the code you are trying to use with `timeit`? Note, your functions are printing in the middle of the algorithm... you generally don't want to include that in your timings. Printing is extremely slow. — juanpa.arrivillaga, Apr 04 '19 at 19:30
I need to time my functions, get the run time as output, then plot them in a graph comparing the runtime against number of iterations for different functions. Also if you're referring to the `sys.std.write` that's just a "progress" that makes my life easier when I fix my code. — user3613025, Apr 04 '19 at 19:37
... the `timeit` module everything you need to get the timings. Have you read the docs? The question? And again, if that is making your life "easier", but it is making your timings *wrong*, or rather, it is *significantly negatively affecting the runtime of that function*, you would want to avoid it because one would want to do an apples-to-apples comparison but you aren't (one function has side effects, printing, the other doesn't). — juanpa.arrivillaga, Apr 04 '19 at 19:45
Like I said, printing is just there for the time being, and will be removed later. I didn't that was hard to understand. — user3613025, Apr 04 '19 at 19:49

Is there a better way to do run time analysis than this?

0 Answers0