13

I'm just reviewing some basics of Python and there's a tricky problem about comparing floating point numbers.

2.2 * 3.0 == 6.6
3.3 * 2.0 == 6.6

I thought these should both return a False. However, the second one gave me a True. enter image description here

Please help me here. Thanks!

Zhiya
  • 610
  • 2
  • 7
  • 22
  • 10
    You shall never compare two float numbers by `==`. Use `abs(a-b) < Threshold` if you really want to. – Steve Sep 29 '14 at 02:22
  • @Steve Yes, I am aware of that, but still thanks for your note :) – Zhiya Sep 29 '14 at 02:29
  • 3
    @Steve: A correct comparison is more complicated than this, since the precision of floats is a number of digits, not an absolute numerical value. Something like `abs(a-b) <= rel_prec * max(abs(a), abs(b))` is better (with rel_prec close to 1e-16, for instance, for Python's double precision floats). In addition to this, the case of a zero value should be handled too. I did not fully check this, but the following might work: `abs(a-b) <= rel_prec * (max(abs(a), abs(b)) if a != 0 != b else 1)`. – Eric O. Lebigot Sep 29 '14 at 02:47
  • 1
    This isn't specific to Python. The same problem occurs for floats in any language. – John La Rooy Sep 29 '14 at 03:28
  • 1
    I can understand expecting them to both return `True` if you didn't understand how floats and rounding errors work, but what could possibly make you expect them both to return `False`? – abarnert Sep 29 '14 at 03:44
  • 4
    @Steve: Huh? You can compare two floating-point numbers for equality using `==`. It works correctly. – tmyklebu Sep 29 '14 at 04:30
  • possible duplicate of [Is floating point math broken?](http://stackoverflow.com/questions/588004/is-floating-point-math-broken) – tmyklebu Sep 29 '14 at 04:30
  • @tmyklebu: That depends on what you mean by "correctly". If you want `2.2 * 3.0 == 6.6` to be `False` because the closest IEEE-double binary floating-point fractions to `2.2` and `3.0` do not multiply to `6.6`, then yes, `==` works correctly. If you want it to be `True` because the real numbers `2.2` and `3.0` do multiply to `6.6`, then `==` does not work correctly. Or, if you want it to be `True` because the real numbers `2.2` and `3.0`, when rounded to binary floats, are within the appropriate propagated error range of `6.6`, then `==` does not work correctly. – abarnert Sep 29 '14 at 04:58
  • @EOL, you are right. The realistic is more complicated. – Steve Sep 29 '14 at 05:05
  • @abarnert: If you want `==` to return `True` for two things that aren't equal, you need to adjust your idea of what `==` should do. – tmyklebu Sep 29 '14 at 05:07
  • @tmyklebu: Which is effectively what Steve said, and you disagreed with. If you want to check whether the results of two floating-point computations are representations of the same number modulo rounding error, you should not be using `==`, because that's not what `==` means (nor what it should mean). – abarnert Sep 29 '14 at 05:39
  • 5
    @abarnert: No, Steve said "don't use `==` to compare two floating-point numbers." That's horrible advice and it enforces unjustified superstitions. – tmyklebu Sep 29 '14 at 05:46
  • 1
    @tmyklebu: OK, in what situation is checking the result of a computation with `==` (e.g., checking `2.2 * 3.0 == 6.6`) a useful thing to do? – abarnert Sep 29 '14 at 06:00
  • 1
    @abarnert: Whenever you want to check for equality of floating-point numbers, you use the operator that checks for equality of floating-point numbers. – tmyklebu Sep 29 '14 at 06:17
  • 3
    @tmyklebu: And when is that? Almost always, when you think you want to check for equality of floating-point numbers, you're wrong, and you do not. If you're in a situation where you really _do_ want to check, you know better. Steve's advice is not superstition or harmful; it's a rule of thumb that people should follow until they run into a situation where they say, "Hold on, here I really care about the N-bit binary fraction" (or "… about the bit pattern" or whatever), at which point they'll understand that rules have exceptions. – abarnert Sep 29 '14 at 07:29
  • 4
    @abarnert: I prefer the rule "understand your tools before using them in serious code" to some hodgepodge of superstitions about `goto` and floating-point numbers. Situations where `==` is appropriate: If you can prove no roundoff error will occur, you use `==`. For instance, Graham's scan can be implemented correctly with `double`s if your points have, say, integer coordinates in [-2^24, 2^24]. The cases where you *can't* use `==` are scarier, since it means you probably need to fall back to MPFR to see whether your predicate is actually true or actually false. – tmyklebu Sep 29 '14 at 13:04

3 Answers3

13

This might be illuminating:

>>> float.hex(2.2 * 3.0)
'0x1.a666666666667p+2'
>>> float.hex(3.3 * 2.0)
'0x1.a666666666666p+2'
>>> float.hex(6.6)
'0x1.a666666666666p+2'

Although they are all displayed in decimal as 6.6, when you inspect the internal representation, two of them are represented in the same way, while one of them is not.

Amadan
  • 191,408
  • 23
  • 240
  • 301
  • 2
    But they're _not_ all displayed in decimal as `6.6`. In 3.x, `print(2.2 * 3.0)` or just `2.2 * 3.0` will show `6.6000000000000005`. In 2.x, the `print` will truncate, but just `2.2 * 3.0` will still show `6.6000000000000005`. So, this is simpler to see than the answer implies. – abarnert Sep 29 '14 at 03:49
3

In order to complete Amadan's good answer, here is a more obvious way of seeing that 2.2*3. and 3.3*2. are not represented by the same float: in a Python shell,

>>> 2.2 * 3.
6.6000000000000005
>>> 3.3 * 2.
6.6

In fact, the Python shell displays the representation of numbers, which by definition should allow the corresponding float to be correctly built back from the representation, so you see the numerical approximation of 2.2*3 that Python does. The fact that 2.2*3. != 3.3*2. is obvious when seeing all the necessary digits, like above.

Eric O. Lebigot
  • 91,433
  • 48
  • 218
  • 260
  • The part about the representation vs. the friendly version is relevant to Python 2.x, but not to 3.x. In 3.x, `print(2.2*3.)` will still give you `6.6000000000000005`. In 3.x, both `str` and `repr` return the shortest string that will evaluate back to the same `float`; in 2.x, `str` truncates to a platform-specific number of digits, while `repr` doesn't. – abarnert Sep 29 '14 at 03:48
  • Interesting. There was no "part about the representation vs the friendly version" before your comment, though. :D – Eric O. Lebigot Sep 29 '14 at 09:44
  • 1
    Reference about Python 3's `str` change: http://stackoverflow.com/questions/25898733/why-does-strfloat-return-more-digits-in-python-3-than-python-2. – Eric O. Lebigot Sep 29 '14 at 11:29
0

It is also obvious that 3.3 * 2.0 is numerically identical to 6.6. The latter computation is nothing more than an increment of the binary exponent as it is the result of a multiplication with a power of two. You can see this in the following:

    | s exponent    significant
----+-------------------------------------------------------------------
1.1 | 0 01111111111 0001100110011001100110011001100110011001100110011010
2.2 | 0 10000000000 0001100110011001100110011001100110011001100110011010
3.3 | 0 10000000000 1010011001100110011001100110011001100110011001100110
6.6 | 0 10000000001 1010011001100110011001100110011001100110011001100110

Above you see the binary representation of the floating point numbers 3.3 and 6.6. The only difference in the two numbers is the exponent since they are only multiplied with two. We know that IEEE-754 will:

  • approximate a decimal number with the smallest numerical error
  • can represent all integers up to 2^53 exactly (for binary64)

So since 2.0 is exactly representable, a multiplication with this number will be nothing more than a change in the exponent. So all the following will create the same floating point numbers:

6.6 == 0.825 * 16.0 == 1.65 * 4.0 == 3.3*2.0 == 13.2 * 0.5 == ...

Does this mean that 2.2*3.0 is different from 6.6 because of the significant? No, this was just due to rounding errors in the multiplication.

An example where it would have worked would have been 5.5*2.0 == 2.2*5.0 == 11.0. Here the rounding was favourable

kvantour
  • 25,269
  • 4
  • 47
  • 72