1

In many computer language I'm familiar with an expression such as

>>> 1.0 - ((1.0/3.0)*3.0)
0.0

will evaluate to a number close to 0.0 but not exactly to 0.0. In Python it seems to evaluate exactly to 0.0. How does this work?

>>> 0.0 == (1.0 - ((1.0/3.0)*3.0))
True
>>> 0.0 == (1.0 - ((1.0/10.0)*10.0))
True
>>> 1.0 - (0.1 * 10)
0.0
>>> 0.0 == (1.0 - (0.1 * 10))
True

When I look into the Python documentation, I don't see this example explicitly, but it seems to imply that, for example 0.1 * 10 would not equal exactly 1. In fact it says that 0.1 is approximately 0.1000000000000000055511151231257827021181583404541015625

It would be great if someone could explain to me what's happening here.

By the way, the post is sort of the opposite of what I'm asking. That article asks why floating point computations are INACCURATE. I'm asking, rather, why are floating point computations surprisingly/magically ACCURATE?

Jim Newton
  • 594
  • 3
  • 16
  • 1
    Note all of these have exactly the same behaviour in Java thereby rendering the first statement not true. The end of the question is still sensible so: why does it work? – luk2302 Aug 10 '23 at 07:22
  • 1
    And in Ruby and JavaScript, and I would suspect in many other languages as well. For a "triggering" test, try `0.1 + 0.2`. – Amadan Aug 10 '23 at 07:22
  • [How does Python convert floats to strings internally?](https://stackoverflow.com/questions/69803845/how-does-python-convert-floats-to-strings-internally) is almost the same question, and has some useful links, the question is not explicitly answered. But it is not that Python is particularly accurate, just that when it converts `3602879701896397 / 36028797018963968` to string, it manages to produce `"0.1"`, and not a more awkward string. – Amadan Aug 10 '23 at 07:42
  • 1
    Python just wraps the platform's `double` type, except that division by zero causes an exception and maybe some similar nuances that I'm not aware of. Another "triggering" example of this would be `10./3. != 1./3. * 10.` – Homer512 Aug 10 '23 at 08:07
  • Your original example also works, but not with 3. Try `1. / 49. * 49.` – Homer512 Aug 10 '23 at 08:18
  • I think I see what's happening. Maybe. If I multiply 0.1 by 10, in IEEE arithmetic, it just increments the exponent, but copies the mantissa. IEEE expresses the mantissa and exponent as base 2 BUT represents the number as a * 10^b, not as a*2^b. However, if I add 0.1 to itself 10 times, I get something that is not equal to 1.0. As expected. However, it does not explain why 1.0 == (1.0 / 3.0) * 3.0 – Jim Newton Aug 10 '23 at 08:18
  • 1
    It's not that the calculation was done correctly, but rather the inaccurate calculation just happened to yield the correct value. Note that `(0.9999999999999999 / 3) * 3` is also `1.0`. – ken Aug 10 '23 at 08:29
  • @JimNewton Why do you *expect* them to be wrong different though? `1.0/3.0 * 3.0` needs to be represented by some discrete floating point value. Python calculates this expression and checks what floating point number it is closest to. If that just happens to be the same value as `1.0`, you will get true equality. – matszwecja Aug 10 '23 at 08:48
  • I don't follow your logic with the base 10 representation. However, The famous [What every computer scientist should know about floating-point arithmetic](https://dl.acm.org/doi/pdf/10.1145/103162.103163) paper discusses exactly your test case in theorem 7 including conditions where it works and where it doesn't – Homer512 Aug 10 '23 at 08:50
  • @Homer512 If x = a * 10^b, then x*10 is simply a*10^(b+1). Thus a, the mantissa, is exactly the same. This is true regardless of the base a and b are represented in. – Jim Newton Aug 10 '23 at 09:29
  • You say it should work regardless of base. Try base 49: `x = a * 49**b`. Then `x*49 = a*49**(b+1)`. However, for `a=1` and `b=-1` this does not work (`1./49. * 49. != 1.`) – Homer512 Aug 10 '23 at 09:55
  • 2
    @JimNewton: Re “IEEE expresses the mantissa and exponent as base 2 BUT represents the number as a * 10^b, not as a*2^b”: What? No, it does not. The IEEE-754 binary formats represent numbers as (-1)^s • f • 2^e, where s is a sign bit, f is a significand, and e is an exponent. Any display of them as decimal is done by converting the number to a decimal form for display; it is not a part of the IEEE-754 binary format or arithmetic. IEEE 754 does specify decimal formats, but those are a different matter and are not what you are using in Python. – Eric Postpischil Aug 10 '23 at 12:02
  • @EricPostpischil, Thank you for the correction. – Jim Newton Aug 11 '23 at 06:50

1 Answers1

0

The lack of any error in the example you provided is non-consequential - you simply chose values that do not happen to trigger a floating point arithmetic error.

import struct
def binary(num):
    return ''.join('{:0>8b}'.format(c) for c in struct.pack('!d', num))

print(f"{binary(0.1 + 0.2) = }")
print(f"{binary(0.3)       = }")

print(f"{binary(0.1 * 10)  = }")
print(f"{binary(1.0)       = }")

Running that code you can easily notice that first pair has 2 different binary representations - one ends with 0011, while the other is the following binary number, ending with 0100. Such a difference does not happen with values around one, as both are approximated by the same value, 1 * 2^0

matszwecja
  • 6,357
  • 2
  • 10
  • 17
  • Re “Such a difference does not happen with values around one”: `1./49*49` produces “0.9999999999999999” (in common Python implementations). – Eric Postpischil Aug 10 '23 at 11:58