6

If as the input you provide the (integer) power, what is the fastest way to create the corresponding power of ten? Here are four alternatives I could come up with, and the fastest way seems to be using an f-string:

from functools import partial
from time import time
import numpy as np

def fstring(power):
    return float(f'1e{power}')

def asterisk(power):
    return 10**power

methods = {
    'fstring': fstring,
    'asterisk': asterisk,
    'pow': partial(pow, 10),
    'np.pow': partial(np.power, 10, dtype=float)
}

# "dtype=float" is necessary because otherwise it will raise: 
# ValueError: Integers to negative integer powers are not allowed.
# see https://stackoverflow.com/a/43287598/5472354
powers = [int(i) for i in np.arange(-10000, 10000)]
for name, method in methods.items():
    start = time()
    for i in powers:
        method(i)
    print(f'{name}: {time() - start}')

Results:

fstring: 0.008975982666015625
asterisk: 0.5190775394439697
pow: 0.4863283634185791
np.pow: 0.046906232833862305

I guess the f-string approach is the fastest because nothing is actually calculated, though it only works for integer powers of ten, whereas the other methods are more complicated operations that also work with any real number as the base and power. So is the f-string actually the best way to go about it?

Chris
  • 26,361
  • 5
  • 21
  • 42
mapf
  • 1,906
  • 1
  • 14
  • 40
  • 2
    *Something* is calculated for the f-string. The key is that the calculations are all happening in C. – chepner Oct 31 '21 at 16:28
  • 1
    One difference is that your `fstring` is just calculating a single `float`, estimating the power of ten in a single `double` type. All others are building a large integer. `sys.getsizeof(10**10000)` gives me 4456 - that's a large integer. I haven't peeked at the algorithm, but gowing a python integer is a relatively expensive operation. If the float is good enough for you application, then use it. But if you really want integers, you pay the price. – tdelaney Oct 31 '21 at 16:46
  • 1
    Have you tried `int('1' + power * '0')`? – Arne Oct 31 '21 at 17:01
  • @Arne since I am also interested in negative powers, this won't work. – mapf Oct 31 '21 at 17:09
  • @chepner well I guess the machine is always doing some kind of computation at some level, but I see that my assumption was too naive. Your explanation makes a lot of sense. Thanks! – mapf Oct 31 '21 at 17:12
  • 2
    The `np.pow` method looks bogus because `float32/float64/float128` will just return `inf` for exponents > `38/308/4932`. – ekhumoro Oct 31 '21 at 17:12
  • @tdelaney that's very interesting. I think for my purposes a `float` is good enough. – mapf Oct 31 '21 at 17:12
  • @ekhumoro ohh that's interesting. I didn't check that. But now it makes a whole lot more sense why I got a `RuntimeWarning: overflow encountered`, which I simply ignored. – mapf Oct 31 '21 at 17:39
  • 1
    `decimal.Decimal` was pretty quick. I did `ten = decimal.Decimal('10')` globally then the function `return ten ** power`. – tdelaney Oct 31 '21 at 18:34

3 Answers3

6

You're comparing apples to oranges here. 10 ** n computes an integer (when n is non-negative), whereas float(f'1e{n}') computes a floating-point number. Those won't take the same amount of time, but they solve different problems so it doesn't matter which one is faster.

But it's worse than that, because there is the overhead of calling a function, which is included in your timing for all of your alternatives, but only some of them actually involve calling a function. If you write 10 ** n then you aren't calling a function, but if you use partial(pow, 10) then you have to call it as a function to get a result. So you're not actually comparing the speed of 10 ** n fairly.

Instead of rolling your own timing code, use the timeit library, which is designed for doing this properly. The results are in seconds for 1,000,000 repetitions (by default), or equivalently they are the average time in microseconds for one repetiton.

Here's a comparison for computing integer powers of 10:

>>> from timeit import timeit
>>> timeit('10 ** n', setup='n = 500')
1.09881673199925
>>> timeit('pow(10, n)', setup='n = 500')
1.1821871869997267
>>> timeit('f(n)', setup='n = 500; from functools import partial; f = partial(pow, 10)')
1.1401332350014854

And here's a comparison for computing floating-point powers of 10: note that computing 10.0 ** 500 or 1e500 is pointless because the result is simply an OverflowError or inf.

>>> timeit('10.0 ** n', setup='n = 200')
0.12391662099980749
>>> timeit('pow(10.0, n)', setup='n = 200')
0.17336435099969094
>>> timeit('f(n)', setup='n = 200; from functools import partial; f = partial(pow, 10.0)')
0.18887039500077663
>>> timeit('float(f"1e{n}")', setup='n = 200')
0.44305286100097874
>>> timeit('np.power(10.0, n, dtype=float)', setup='n = 200; import numpy as np')
1.491982370000187
>>> timeit('f(n)', setup='n = 200; from functools import partial; import numpy as np; f = partial(np.power, 10.0, dtype=float)')
1.6273324920002779

So the fastest of these options in both cases is the obvious one: 10 ** n for integers and 10.0 ** n for floats.

kaya3
  • 47,440
  • 4
  • 68
  • 97
  • 1
    I'd say their use of `partial` corresponds to putting their "expression solutions" into functions, that seems rather fair. Unlike your comparison of the expressions with extra `partial`ed calls. Better drop the `partial`, or try both with and without. – Kelly Bundy Oct 31 '21 at 18:40
  • @KellyBundy I'm not sure that was OP's intention; something like `def using_pow(n): return pow(10, n)` would have been clearer about that. Nonetheless you make a good point, I've edited to include straightforward `pow` calls. – kaya3 Oct 31 '21 at 18:43
  • 1
    Well, their *real* intention looks clear: find the fastest way to get the result given a power. So yeah, best try both ways. – Kelly Bundy Oct 31 '21 at 18:55
  • Hey @kaya3, thanks for your answer! Kelly is right, that was my intention. The use of partial was simply an attempt to provide a fair playground. On second thought though, I didn't even need `partial`. I could have just as well used `lambda`, e.g. `lambda power: pow(10, power)`, or "properly" defined those functions like the other ones. – mapf Oct 31 '21 at 19:06
  • I don't think I compare apples to oranges though as you say, because what I am looking for is simply the fastest way to calculate powers of ten, be the results `floats` or `integers`. I'm not sure what you mean by `10 ** n` computes an integer, because e.g. `10 ** -8` is absolutely possible. On the other hand, for the f-string, I *have* to use `float(f'1e{power}')`, because `int(f'1e{power}')` would obviously not work for negative powers. – mapf Oct 31 '21 at 19:09
  • Regarding integers vs. floats, I wrote *"(when n is non-negative)"*. – kaya3 Oct 31 '21 at 19:20
  • [faster float solution](https://tio.run/##PU9BbsQgDLzzirmsCBFd0eXSrdqXRFHVA7RIG0DgHPr6FC9JfLA1nvGMnP/oN0X7lsu2@ZIWUFhcIIQlp0I7EqI6WjM@IaUUsc2bMcK3Ob2aq8E4IsCn0nqImMbyHX/cYM1daezgxd6shlHzLNhD1PRYKaRY2USglTysotR9wTh4RHyAz@Ee1cFPcW6CWQgO/OLAPU69P894fbgzeyZ1motaaP9sOFiN54vq1OQSIg3ycrUea4XEBaRPs67rGrVt/w)? – Kelly Bundy Oct 31 '21 at 19:21
  • @KellyBundy Interesting way of doing this, but it will give incorrect results for numbers between 309 and 632. If you check for that then it's still faster, though. If you write an answer then I will upvote it. – kaya3 Oct 31 '21 at 19:26
  • @kaya3 sorry, fair enough! That's pretty interesting though. Does that mean internally the `**` operator first checks if the exponent is negative (and integer I suppose) and then executes different routines depending on the state? – mapf Oct 31 '21 at 19:27
  • @kaya3 Ah right. I ignored that because its competitor `10.0 ** i` can't handle larger `i`, either. But yes, crashing with a proper error message is better than giving a wrong result. – Kelly Bundy Oct 31 '21 at 19:28
  • @KellyBundy I second kaya3, though in my case it's also because I'm not really sure what that's supposed to tell me. – mapf Oct 31 '21 at 19:29
  • 1
    @mapf Yes, it checks if the exponent is negative and uses a different routine for floating-point powers in that case; [source code link](https://github.com/python/cpython/blob/main/Objects/longobject.c#L4198). – kaya3 Oct 31 '21 at 19:33
  • @kaya3 I see, thanks! One (probably) last thing, because I'm not familiar with `timeit`: it looks like you are only testing positive powers, do you think negative powers won't have an effect? – mapf Oct 31 '21 at 19:42
  • @mapf I tested positive powers only to demonstrate how you should measure the times more accurately. For negative powers there should be no significant difference between `10 ** n` and `10.0 ** n` since the former just calls the latter (as seen in the source code). – kaya3 Oct 31 '21 at 19:46
2

Another contender for the floats case, precompute all possible nonzero finite results and look them up:

0.0 if n < -323 else f[n] if n < 309 else inf

The preparation:

f = [10.0 ** i for i in [*range(309), *range(-323, 0)]]
inf = float('inf')

Benchmark with kaya3's exponent n = 200 as well as n = -200 as negative exponent with nonzero result and n = -5000 / n = 5000 as medium-size negative/positive exponents from your original range:

n = 200
487 ns  487 ns  488 ns  float(f'1e{n}')
108 ns  108 ns  108 ns  10.0 ** n
128 ns  129 ns  130 ns  10.0 ** n if n < 309 else inf
 72 ns   73 ns   73 ns  0.0 if n < -323 else f[n] if n < 309 else inf

n = -200
542 ns  544 ns  545 ns  float(f'1e{n}')
109 ns  109 ns  110 ns  10.0 ** n
130 ns  130 ns  131 ns  10.0 ** n if n < 309 else inf
 76 ns   76 ns   76 ns  0.0 if n < -323 else f[n] if n < 309 else inf

n = -5000
291 ns  291 ns  291 ns  float(f'1e{n}')
 99 ns   99 ns  100 ns  10.0 ** n
119 ns  120 ns  120 ns  10.0 ** n if n < 309 else inf
 34 ns   34 ns   34 ns  0.0 if n < -323 else f[n] if n < 309 else inf

n = 5000
292 ns  293 ns  293 ns  float(f'1e{n}')
 error   error   error  10.0 ** n
 33 ns   33 ns   33 ns  10.0 ** n if n < 309 else inf
 53 ns   53 ns   53 ns  0.0 if n < -323 else f[n] if n < 309 else inf

Benchmark code (Try it online!):

from timeit import repeat

solutions = [
    "float(f'1e{n}')",
    '10.0 ** n',
    '10.0 ** n if n < 309 else inf',
    '0.0 if n < -323 else f[n] if n < 309 else inf',
]

for n in 200, -200, -5000, 5000:
    print(f'{n = }')
    setup = f'''
n = {n}
f = [10.0 ** i for i in [*range(309), *range(-323, 0)]]
inf = float('inf')
'''
    for solution in solutions:
        try:
            ts = sorted(repeat(solution, setup))[:3]
        except OverflowError:
            ts = [None] * 3
        print(*('%3d ns ' % (t * 1e3) if t else ' error ' for t in ts), solution)
    print()
Kelly Bundy
  • 23,480
  • 7
  • 29
  • 65
  • Geat, thanks a lot for writing out your suggestion! It's a really sneaky idea. Though as it's still based on `10.0 ** i`, I think it's still fair to accept kaya3's answer after all. – mapf Oct 31 '21 at 19:57
  • @mapf Well, I could base it on *any* solution, as I'm disregarding the preparation anyway :-). But yes, accept whatever answer you like best, and kaya3's surely is good, as usual. – Kelly Bundy Oct 31 '21 at 20:02
  • Hi, I thought about this problem again, and realized that a `dict` should be even faster for the lookup, however, [it doesn't seem to be the case](https://stackoverflow.com/q/69838324/5472354). Do you know why? – mapf Nov 04 '21 at 11:12
  • @mapf And now you know why I used that somewhat tricky list construction with two ranges instead of the simpler dict :-) (I think I had considered dict for a brief moment but dismissed it because it's slower). As the answer there says, you're confusing lookups with lookups (as in, confusing accessing with searching). – Kelly Bundy Nov 04 '21 at 11:35
  • Yes, thanks! I wasn't aware that the term "lookup" was ambiguous, I feel like that should have been clarified better. In any case, my question is getting downvoted for some reason, so I might just delete it... – mapf Nov 04 '21 at 11:39
0

You could try it with a logarithmic approach using math.log and math.exp but the range of values will be limited (which you can handle with try/except).

This seems to be just as fast as fstring if not a bit faster.

import math
ln10 = math.log(10)
def mPow(power):
    try:
        return math.exp(ln10*power)
    except:
        return 0 if power<0 else math.inf

[EDIT] Given that we are constrained by the capabilities of floats, we might as well just prepare a list with the 617 possible powers of 10 (that can be held in a float) and get the answer by index:

import math
minP10,maxP10 = -308,308
powersOf10 = [10**i for i in range(maxP10+1)]+[10**i for i in range(minP10,0)]
def tenPower(power):
    if power < minP10: return 0
    if power > maxP10: return math.inf
    return powersOf10[power]    # negative indexes for powers -308...-1

This one is definitely faster than fstring

Alain T.
  • 40,517
  • 4
  • 31
  • 51
  • Interesting idea! Is there no way to also make it work with negative powers? – mapf Oct 31 '21 at 18:44
  • It should work with negative powers as well (at least it did in my tests) – Alain T. Oct 31 '21 at 20:28
  • I'm not sure I understand. You return 0 if the power is < 0, no? – mapf Oct 31 '21 at 20:32
  • Only when `math.exp(ln10*power)` fails – Alain T. Oct 31 '21 at 20:34
  • Ahh, ok. My bad. I thought it would always fail if power < 0. Neat! I ran my (imperfect) speed test, and it looks like it really is quite fast. Judging from my numbers the fastest even. – mapf Oct 31 '21 at 20:36
  • Going through a logarithm and exponentiation introduces a bit of imprecision. See my last \[EDIT\] for a solution that is more precise and even faster. – Alain T. Oct 31 '21 at 21:06
  • Hey, I see! Using a look-up list is pretty nifty. It's also what Kelly Bundy was proposing. – mapf Nov 01 '21 at 08:01