c++ additive identity unsafe example ( a+0.0 != a )

Question

In MSDN article, it mentions when fp:fast mode is enabled, operations like additive identity (a±0.0 = a, 0.0-a = -a) are unsafe. Is there any example that a+0 != a under such mode?

EDIT: As someone mentioned below, this sort of issue normally comes up when doing comparison. My issue is from comparison, the psedocode looks like below:

for(i=0;i<v.len;i++)
{
  sum+=v[i];
  if( sum >= threshold) break;
}

It breaks after adding a value of 0 (v[i]). The v[i] is not from calculation, it is assigned. I understand if my v[i] is from calculation then rounding might come into play, but why even though I give v[i] a zero value, I still have this sum < threshold but sum + v[i] >= threshold?

If you haven't done so before, you may want to read [the Stack Overflow question checklist](http://meta.stackexchange.com/questions/156810/stack-overflow-question-checklist). — Some programmer dude, Oct 17 '13 at 17:14
See: http://stackoverflow.com/questions/16477037/why-does-msvs-not-optimize-away-0-insted-it-turns-it-into-a-denormalized-float and http://stackoverflow.com/questions/9314534/why-does-changing-0-1f-to-0-slow-down-performance-by-10x — Mysticial, Oct 17 '13 at 17:20
What are the types of `sum` and `v[i]` here? `float` or `double`? — Mark Dickinson, Oct 17 '13 at 18:51

score 3 · Answer 1 · answered Oct 17 '13 at 17:37

The reason that it's "unsafe" is that what the compiler assumes to be zero may not really end up being zero, due to rounding errors.

Take this example which adds two floats on the edge of the precision which 32 bit floats allows:

    float a = 33554430, b = 16777215;
    float x = a + b;
    float y = x - a - b;
    float z = 1;
    z = z + y;

With fp:fast, the compiler says "since x = a + b, y = x - a - b = 0, so 'z + y' is just z". However, due to rounding errors, y actually ends up being -1, not 0. So you would get a different result without fp:fast.

That's a good example, but for a different optimization, namely (a+b) - a - b -> 0. The OP was asking about a + 0.0 -> a, not "a + (something inferred to be 0) -> a". There's a literal 0.0 there. — c-urchin, Oct 17 '13 at 18:35

score 2 · Answer 2 · answered Oct 17 '13 at 17:27

It's not saying something 'fixed' like, "if you set /fp:fast, and variable a happens to be 3.12345, then a+0 might not be a". It's saying that when you set /fp:fast, the compiler will take shortcuts that mean that if you compute a+0, and then compare that to what you stored for a, there is no guarantee that they'll be the same.

There is a great write up on this class of problems (which are endemic to floating point calculations on computers) here: http://www.parashift.com/c++-faq-lite/floating-point-arith2.html

score 2 · Answer 3 · answered Oct 17 '13 at 18:16

2

If a is -0.0, then a + 0.0 is +0.0.

answered Oct 17 '13 at 18:16

dan04

87,747
23
163
198

Agree, but -0.0 == 0.0 evaluates to True also. – Steven Oct 17 '13 at 18:35
1

This begs the question, when can replacing -0.0 by +0.0 affect the result of a calculation? It can make the difference between +Inf and -Inf if used in division. But assuming no exceptions (ie NaN or +-Inf) is there a calculation where switching +0.0 and -0.0 would matter? – c-urchin Oct 17 '13 at 18:41
1

@c-urchin: it matters in some transcendental and complex functions. For example under some language standards `atan2(-0.0, -1.0)` gives `-pi`, while `atan2(0.0, -1.0)` gives `pi`. – Mark Dickinson Oct 17 '13 at 18:45

bames53 · Accepted Answer · 2013-10-30T17:17:17.420

it mentions when fp:fast mode is enabled, operations like additive identity (a±0.0 = a, 0.0-a = -a) is unsafe.

What that article says is

Any of the following (unsafe) algebraic rules may be employed by the optimizer when the fp:fast mode is enabled:

And then it lists a±0.0 = a and 0.0-a = -a

It is not saying that these identities are unsafe when fp:fast is enabled. It is saying that these identities are not true for IEEE 754 floating point arithmetic but that /fp:fast will optimize as though they are true.

I'm not certain of an example that shows that a + 0.0 == a to be false (except for NaN, obviously), but IEEE 754 has a lot of subtleties, such as when intermediate values should be truncated. One possibility is that if you have some expression that includes + 0.0, that might result in a requirement under IEEE 754 to do truncation on an intermediate value, but that /fp:fast will generate code that doesn't do the truncation, and consequently later results may differ from what is strictly required by IEEE 754.

Using Pascal Cuoq's info here's a program that produces different output based on /fp:fast

#include <cmath>
#include <iostream>

int main() {
    volatile double a = -0.0;
    if (_copysign(1.0, a + 0.0) ==  _copysign(1.0, 0.0)) {
        std::cout << "correct IEEE 754 results\n";
    } else {
        std::cout << "result not IEEE 754 conformant\n";
    }
}

When built with /fp:fast the program outputs "result not IEEE 754 conformant" while building with /fp:strict cause the program to output "correct IEEE 754 results".

Besides Nan, the equality `a + 0.0 == a` fails when `a` is `-0.0`. -0.0 + 0.0 evaluates to +0.0 in round-to-nearest-even. — Pascal Cuoq, Oct 17 '13 at 23:02
@c-urchin You should tell the author of this answer that his uncertainty has been answered then, not me. This answer still expresses doubt about “an example that shows that …”, hence my comment. This said, you are right, I am wasting my time. — Pascal Cuoq, Oct 18 '13 at 19:35
@PascalCuoq The expression `-0.0 + 0.0 == -0.0` does actually evaluate to true. Is there a way to use negative zero to generate a value that compares as not-equal with the same expression using positive zero? — bames53, Oct 18 '13 at 20:15
@bames53 `1.0 / (+0.0)` evaluates to `+inf`, but `1.0 / (-0.0)` evaluates to `-inf`. Also `copysign(1.0, +0.0) != copysign(1.0, -0.0)` — Pascal Cuoq, Oct 18 '13 at 21:40

c++ additive identity unsafe example ( a+0.0 != a )

4 Answers4