Python epsilon is not the smallest number

Question

What does sys.float_info.epsilon return?

On my system I get:

>>> sys.float_info.epsilon
2.220446049250313e-16
>>> sys.float_info.epsilon / 2
1.1102230246251565e-16
>>> 0 < sys.float_info.epsilon / 2 < sys.float_info.epsilon
True

How is this possible?

EDIT:

You are all right, I thought epsilon does what min does. So I actually meant sys.float_info.min.

EDIT2

Everybody and especially John Kugelman, thanks for your answers!

Some playing around I did to clarify things to myself:

>>> float.hex(sys.float_info.epsilon)
'0x1.0000000000000p-52'
>>> float.hex(sys.float_info.min)
'0x1.0000000000000p-1022'
>>> float.hex(1 + a)
'0x1.0000000000001p+0'
>>> float.fromhex('0x0.0000000000001p+0') == sys.float_info.epsilon
True
>>> float.hex(sys.float_info.epsilon * sys.float_info.min)
'0x0.0000000000001p-1022'

So epsilon * min gives the number with the smallest positive significand (or mantissa) and the smallest exponent.

score 5 · Accepted Answer · edited May 23 '17 at 11:58

epsilon is the difference between 1 and the next representable float. That's not the same as the smallest float, which would be the closest number to 0, not 1.

There are two smallest floats, depending on your criteria. min is the smallest normalized float. The smallest subnormal float is min * epsilon.

>>> sys.float_info.min
2.2250738585072014e-308
>>> sys.float_info.min * sys.float_info.epsilon
5e-324

Note the distinction between normalized and subnormal floats: min is not actually the smallest float, it's just the smallest one with full precision. Subnormal numbers cover the range between 0 and min, but they lose a lot of precision. Notice that 5e-324 has only one significant digit. Subnormals are also much slower to work with, up to 100x slower than normalized floats.

>>> (sys.float_info.min * sys.float_info.epsilon) / 2
0.0
>>> 4e-324
5e-324
>>> 5e-325
0.0

These tests confirm that 5e-324 truly is the smallest float. Dividing by two underflows to 0.

See also: What is the range of values a float can have in Python?

My example works the same way for sys.float_info.min instead of sys.float_info.epsilon everywhere. How do you explain this? — Ella Sharakanski, Apr 17 '15 at 21:34
Anything smaller than `min` is subnormal. `min * epsilon` is the true smallest float. If you divide it by 2 it underflows to 0. — John Kugelman, Apr 17 '15 at 21:41

score 1 · Answer 2 · answered Apr 17 '15 at 21:31

1

You actually want sys.float_info.min ("minimum positive normalized float"), which on machine gives me .2250738585072014e-308.

epsilon is:

difference between 1 and the least value greater than 1 that is representable as a float

See the docs for more info on the fields of sys.float_info.

answered Apr 17 '15 at 21:31

Isaac

15,783
9
53
76

`min` still isn't quite the smallest positive float; there are still subnormals below it. – user2357112 Apr 17 '15 at 21:33
Thanks, you are right, but how do you explain what @user2357112 said? – Ella Sharakanski Apr 17 '15 at 21:35
1

Subnormals floats mean the number has zeros in its significand; I think @John Kugelman has the better answer at this point: I'd accept that one if it works for you, otherwise I can update this one if it doesn't. – Isaac Apr 17 '15 at 22:00

score 0 · Answer 3 · answered Apr 17 '15 at 21:29

0

Your last expression is possible, because for any real, positive number, 0 < num/2 < num.

From the docs:

difference between 1 and the least value greater than 1 that is representable as a float

answered Apr 17 '15 at 21:29

A.J. Uppal

19,117
6
45
76

score 0 · Answer 4 · answered Apr 17 '15 at 21:29

0

sys.float_info is defined as

difference between 1 and the least value greater than 1 that is representable as a float

on this page.

answered Apr 17 '15 at 21:29

Mike Pelley

2,939
22
23

score 0 · Answer 5 · answered Apr 17 '15 at 21:31

The documentation defines sys.float_info.epsilon as the

difference between 1 and the least value greater than 1 that is representable as a float

However, the gap between successive floats is bigger for bigger floats, so the gap between epsilon and the next smaller float is a lot smaller than epsilon. In particular, the next smaller float is not 0.

score 0 · Answer 6 · answered Apr 17 '15 at 21:35

Like every answer says, it's the difference between 1 and the next greatest value that can be represented, if you tried to add half of it to 1, you'll get 1 back

>>> (1 + (sys.float_info.epsilon/2)) == 1
True

Additionally if you try to add two thirds of it to 1, you'll get the same value:

>>> (1 + sys.float_info.epsilon) == (1 + (sys.float_info.epsilon * (2./3)))
True

Python epsilon is not the smallest number

6 Answers6

Linked