1

My question is both specific (see example) and general. I am looking for best practices for writing fast-running python code. Of two or more ways of evaluating an expression or constructing a statement, I want to have an educated guess as to which will run faster before I do timeit comparison tests.

For example:

xx = [1,2,3,4,5]
x_bar = int(sum(xx)/len(xx)) # or = np.mean(xx)
x_c = [int(x_i - x_bar) for x_i in xx]
TSS = sum([x_i*x_i for x_i in x_c])

I thought I might speed things up by changing x_i*x_i to x_i**2. But if I am reading the results correctly, it was actually ~3 times slower:

%%timeit
sum([x_i*x_i for x_i in x_c])
# 670 ns ± 19 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

%%timeit
sum([x_i**2 for x_i in x_c])
# 1.8 µs ± 120 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

An explanation of why this is true for the * operator vs. the ** operator will be helpful. Even more helpful will be advice on principles, rules of thumb, and resources that will help me recognize opportunities to improve my code. Any suggestions?

Karl Baker
  • 903
  • 12
  • 27
  • 1
    It sounds like the peephole optimizer doesn't recognize that `x_i**2` is `x_i*x_i`, and so is using the general algorithm `exp(2*log(x_i))` instead. In general, `x ** y` is *not* simply repeated multiplication. – chepner Aug 08 '18 at 20:42
  • I think you are saying that the compiler is not optimizing as you would expect it to. But I'm unclear which expression you would expect to run faster, `x_i**2` or `x_i*x_i`. Or would you expect the compiler to optimize them both to the same algorithm? – Karl Baker Aug 08 '18 at 21:54
  • this answer https://stackoverflow.com/a/18453999/8069403 is quite interesting, the following discussion about compiler too – xdze2 Aug 08 '18 at 22:03
  • Thanks. For low powers multiplication is faster, but as the power goes up, exponentiation becomes faster than a string of multiplications. I think I need to look at the bytecode for each of these to see what is different. – Karl Baker Aug 08 '18 at 23:23

0 Answers0