Finding the smallest float in vanilla Python

Question

I was trying to run some little numerical experiments in Python, and I ran into some unexpected behavior. I wrote a program to find the smallest (positive) float in standard Python. Here it is:

x = 1.0

while True :
    y = x
    x /= 2.0
    if x == 0:
        break

print(y)

If I'm not mistaken, vanilla Python uses double precision floats, and the smallest (positive) number should be 2^(-1022) (or 2^(-1023) or 2^(-1024), not quite sure).

However, my program returned something very different. It returns 5e-324.

Now, if we want to turn 2^(-1022) into the form of 5*10^(x), we have to solve the equation 2^(-1022) = 5*10^(x), whose solution is log [2^(-1022)/5] (where the logarithm is base 10).

However, the solution in WolframAlpha is about -308, which is pretty far from -324. I tried using 2^(-1023) and 2^(-1024), and the result is still about -308.

Can you please help me understand this discrepancy? Thanks, any help is appreciated!

Does this answer your question? [What is the range of values a float can have in Python?](https://stackoverflow.com/questions/1835787/what-is-the-range-of-values-a-float-can-have-in-python) — deadshot, Jun 12 '20 at 21:29
It's probably a [denormalized (denormal) number](https://en.wikipedia.org/wiki/Denormal_number). — Tom Karzes, Jun 12 '20 at 21:32
Yet more info on [denormal numbers](https://www.johndcook.com/blog/2009/04/06/anatomy-of-a-floating-point-number/#:~:text=The%20smallest%20denormalized%20positive%20number,number%20must%20underflow%20to%20zero.) — Tom Karzes, Jun 12 '20 at 21:33
@Blue Look at the links I posted, they'll be more helpful. It's a denormal number. — Tom Karzes, Jun 12 '20 at 21:35
@TomKarzes Oh I see, that makes sense. Would you know how to disable denormal numbers in Python and have just plain double precision floats? I am doing these experiments to put into practice my theoretical knowledge of floats. — Blue, Jun 12 '20 at 21:35
@Blue I'm not sure how much control is available. The underlying hardware will produce denormal numbers when it has no choice. The best you could do would be to force them to something like zero, or the smallest normal number. — Tom Karzes, Jun 12 '20 at 21:36
Python won't give you any control over normal/denormal floats - the processor just does it automatically as needed. — Mark Ransom, Jun 12 '20 at 21:47

Mark Ransom · Accepted Answer · 2020-06-13T02:23:18.787

You can't prevent Python from generating a denormal float, but you can detect when it has done so by looking at the bit pattern of the result. If all the exponent bits are zero, you have a denormal. You can see the IEEE-754 binary64 format on Wikipedia. Since bitwise operations like & don't work on a float, the combination of struct.pack and struct.unpack are used to obtain the bit pattern as an integer.

import struct

def is_denormal(x):
    bit_pattern = struct.unpack('q', struct.pack('d', x))[0]
    return (bit_pattern & 0x7ff0000000000000) == 0

x = 1.0
while True :
    y = x
    x /= 2.0
    if is_denormal(x):
        break

print(y)
2.2250738585072014e-308

@TomKarzes that was poorly worded. What I meant was to move the bit-pattern of the double into an integer so it could be checked, because you can't do logical operations like `&` on a `float`. — Mark Ransom, Jun 12 '20 at 23:30

Finding the smallest float in vanilla Python

1 Answers1