1

Some time ago i have posted a question here and I got to know that floating point values should not be compared with double values due to varying precision and we may not get predictable results always. But recently I stumbled across another code where comparison between two or more floating point numbers also resulted in quite strange behavior.

Here is the code I was came across:

#include<stdio.h>
int main()
{
    float a=0.0f;
    int i;
    for(i=0;i<10;i++)
        a=a+0.1f;
    if(a==1.0f) printf("True\n");
    else printf("False\n");
    a=0.0f;
    for(i=0;i<5;i++)
        a=a+0.2f;
    if(a==1.0f) printf("True\n");
    else printf("False\n");
}

The code gave me False and true as the output which startled me quite a bit. Why is this behavior? If there is a loss of precision as the number 0.1f not being represented precisley in binary representation and adding it time and again is causing the summation to be lesser than 1.0f? The same should be true for the next loop right? How far can we trust the floating point arthimatic?

Community
  • 1
  • 1
sasidhar
  • 7,523
  • 15
  • 49
  • 75
  • adding a `printf()`, `a` is shown as `1.000000` in both cases – Ingo Leonhardt Jul 18 '13 at 16:03
  • All your answers are found here (http://floating-point-gui.de/) – abelenky Jul 18 '13 at 16:04
  • 1
    "How far can we trust the floating point arthimatic?" To within a very small delta. Which is to say not exactly. Never (unless you REALLY understand what you're doing) use `==` to compare any floating point values, doesn't matter whether `float` or `double`. – Hot Licks Jul 18 '13 at 16:10
  • You begin with a falsehood: doubles and floats *can* be directly compared. However, the same care must be used as for comparing floats with each other as well as doubles with each other. – wallyk Jul 18 '13 at 16:14
  • NEVER explicitly trust floating point arithmetic down to the last bit. That least significant bit is always a risk. –  Jul 18 '13 at 16:24
  • 2
    @woodchips: The least significant bit is not always a risk. There are techniques for handling floating-point values well, but they require knowledge and careful application. And higher-level languages generally do not support floating-point well, so it also requires implementation-specific code. – Eric Postpischil Jul 18 '13 at 16:45
  • @woodchips there are cases where errors accumulate so there's more than just the last bit you can't trust. – Mark Ransom Jul 18 '13 at 16:51
  • 1
    @EricPostpischil - Yes, I realize that there are cases where the LSB is NOT at risk. I frequently take advantage of exactly that. But unless you understand the behavior of floating point arithmetic well enough, it is a good idea to NEVER assume. –  Jul 18 '13 at 17:09
  • @MarkRansom - Yes, it is also true that more than one bit can be at risk, and often is. But at the very least, the LSB is often risky business, again unless you understand floating point arithmetic in sufficient depth. –  Jul 18 '13 at 17:11

2 Answers2

2

You assume that a+0.2 is equal to a+0.1+0.1. This is not the case (because of rounding errors) for some values of a, but is the case for others. For example, when a==0 obviously both are equal, but if a is the smallest number where a+0.1==a then obviously both are different.

See this code:

#include<stdio.h>
int main()
{
  float a=0.0f;
  int i;
  for(i=0;i<10;i++){
    if (a+0.1f+0.1f==a+0.2f)
      printf("i=%d, equal!\n",i);
    else
      printf("i=%d, delta=%.10f\n",i,(a+0.1f+0.1f)-(a+0.2f));
    a=a+0.1f;
  }
  return 0;
}

The output is:

i=0, equal!
i=1, equal!
i=2, equal!
i=3, equal!
i=4, equal!
i=5, delta=0.0000000596
i=6, delta=0.0000000596
i=7, delta=0.0000000596
i=8, equal!
i=9, equal!
rabensky
  • 2,864
  • 13
  • 18
1

A floating point number (which includes float and double in C) is represented by two parts, both of which have a fixed number of bits in which to hold their value:

  1. a binary fraction called the mantissa (with no bits to the left of the binary-point and) with no zeros immediately to the right of the binary point. (This compares to a decimal representation of a number. The digits to the left of the decimal point can be compared to bits to the left of a binary point and the fractional digits to the right of the decimal point compare to the fractional binary bits to the right of the binary point).
  2. an exponent that tells what power of 2 to multiply that mantissa by. (Compare this to scientific notation 0.1e5 has 5 as the exponent that tells what power of 10 to multiply the mantissa by)

In decimal, we can't represent the fraction 1/3 with a fixed number of fractional digits. For example, 0.333333 isn't exactly equal to 1/3 as the 3 needs to repeat infinitely.

In binary, we can't represent the fraction 1/10 with a fixed number of fractional bits. In this case the binary number 0.00011001100110011 isn't exactly equal to 1/10 as the 0011 needs to repeat indefinitely. So, when 1/10 is converted to floating point, this part is cut off to fit the available bits.

In binary, any fraction with the denominator (the bottom) divisible by 10 is infinitely repeating. That means that a lot of float values are inexact.

When added they are inexact. If you add a lot of them together, the inexactness may cancel or reinforce depending on what value was in the bits that got chopped off when we turned the infinitely repeating binary fraction into a binary fraction with a fixed number of digits.

You also get inexactitude with large numbers, fractions with a lot of digits or when adding numbers that are very different. For example, 1 billion plus .0000009 can't be represented in the available number of bits so the fraction gets rounded away.

You can see that it gets complicated. In any particular case, you can come up with the floating point representation, evaluate the error due to chopped off bits and rounding when multiplying or dividing. At that point you can see exactly why its wrong if you go to the trouble.

Simplified Example - imprecise representation

Here's an example ignoring the exponent and having the mantissa un-normalized, which means left side zeros aren't removed. (0.0001100 = 1/10 and 0.0011001 = 1/20 when chopped after 7 bits) Note that in the real case the issue happens many more digits to the right:

0.0001100 = 1/10
0.0001100
0.0001100
0.0001100                             0.0011001 = 2/10 (1/5)
0.0001100                             0.0011001
0.0001100                             0.0011001
---------                             ---------
       00 <- sum of right 2 columns          11 <- sum of right column
    11000 <- sum of next column             00  <- sum of next two columns  
   110 <- sum of next column              11 <- sum of next column
  000 <- sum of other columns            11 <- sum of next column
-------                                000 <- sum of other columns
0.1001000 <- sum                      ---------
                                      0.1001011 <- sum

We could have the same problem with fractions like 0.12345678901234567890 that wouldn't fit in the 7 bits of my example.

What to Do

First, keep in mind that floating point numbers may not be exact. Adding or subtracting and, even more, multiplying or dividing should be expected to create inexact results.

Second, when comparing two float (or double) values, it is best to compare the difference to some "epsilon". So if, heaven forbid, you were storing US Dollar calculations in float variables, it would look like this. We don't care about anything less than half a cent:

if (fabsf(f1 - f2) >= 0.005f) ...

This means that the numbers are close to each other and, for your purposes, close enough. (@EricPostpischil points out that there is no general definition of "close enough." It has to do with what your calculations hope to accomplish.)

Doing the comparison to some small value takes care of all the loose bits that may be sitting in the low fractional digits after some floating point arithmetic takes place.

Note that if you compare to a constant, it looks similar:

if (fabsf(f1 - 1.0f) >= 0.000001f) ...

or you could do with two comparisons to check the same range of differences:

if (f1 < 0.999999f || f1 > 1.000001f) ...

I should point out, again, that each problem has its own number of interesting fractional decimal digits.

For example, if Google tells you how far apart two positions on the earth are in Kilometers, you may care to the nearest meter so you say any two positions within 0.001 (a thousandth of a kilometer) are functionally identical. Compare the difference to 0.0005. Or you may only care to the nearest block so compare the difference to 0.03 (300 meters). So compare the difference to 0.015.

The same thing applies when your measuring tools are only so accurate. If you measure with a yardstick, don't expect the result to be accurate to 1/100th of an inch.

Lee Meador
  • 12,829
  • 2
  • 36
  • 42
  • 1
    This answer does not explain why adding .1f ten times produces a different result than adding .2f five times. And it is bad advice to recommend comparing floating-point numbers with a tolerance, because it trades false negatives for false positives (which is inappropriate without knowing the application), the tolerance cannot be determined without information about the preceding operations and data and the application requirements, and other reasons. – Eric Postpischil Jul 18 '13 at 16:23
  • @EricPostpischil If someone is writing code and chooses to use floats or doubles without knowing the tolerances dictated by the problem domain, we need to take away their programming license. I'll add in some info about why floating point arithmetic seems inexact. There was an answer that was deleted that answered that. – Lee Meador Jul 18 '13 at 16:27
  • 1
    The error is not a function of the domain; it is a function of the specific operations and data involved, as well as the type. Although the `double` type has a certain precision, some algorithms are numerically unstable, so they magnify the errors, and some algorithms are stable and minimize the errors. There is no general recommendation for tolerance. – Eric Postpischil Jul 18 '13 at 16:42
  • @EricPostpischil Absolutely right. No general tolerances suffice. Problem domain tolerances matter. Its not simple. You have to determine that whatever calculations you do are generating less error than your problem can tolerate. That is often the case but you can't count on it. – Lee Meador Jul 18 '13 at 16:58