Why this same code produce two different fp results on different Machines?

Question

Here's the code:

#include <iostream>
#include <math.h>

const double ln2per12 = log(2.0) / 12.0;

int main() {
    std::cout.precision(100);
    double target = 9.800000000000000710542735760100185871124267578125;
    double unnormalizatedValue = 9.79999999999063220457173883914947509765625;
    double ln2per12edValue = unnormalizatedValue * ln2per12;
    double errorLn2per12 = fabs(target - ln2per12edValue / ln2per12);
    std::cout << unnormalizatedValue << std::endl;
    std::cout << ln2per12 << std::endl;
    std::cout << errorLn2per12 << " <<<<< its different" << std::endl;
}

If I try on my machine (MSVC), or here (GCC):

errorLn2per12 = 9.3702823278363212011754512786865234375e-12

Instead, here (GCC):

errorLn2per12 = 9.368505970996920950710773468017578125e-12

which is different. Its due to Machine Epsilon? Or Compiler precision flags? Or a different IEEE evaluation?

What's the cause here for this drift? The problem seems in fabs() function (since the other values seems the same).

All sorts of things could be going on. Some machines have more precision then `double` actually uses and you'll get less compound error with the results saying in those extended registers then you would if each operation gets truncated. — NathanOliver, Feb 14 '19 at 15:53
Possible duplicate of [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) — Matthieu Brucher, Feb 14 '19 at 15:55
`-Ofast` turns on `-ffast-math`. Which _[...can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions_](https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html). — Daniel Langr, Feb 14 '19 at 15:56
"Or Compiler precision flags?" - Impossible to answer when you don't tell us *what flags* you pass to your compiler. But, yes, some flags, like `-Ofast` for GCC allows the compiler to break the rules and *can* lead to weird results (don't use it) - the Microsoft compiler has similar dangerous flags. — Jesper Juhl, Feb 14 '19 at 15:56
@MatthieuBrucher: please NO! This is not a duplicate of that question! Its specific per platform. Please read carefully — markzzz, Feb 14 '19 at 15:57
According to [the Wikipedia IEEE754 article](https://en.wikipedia.org/wiki/IEEE_754) , double precision values have 15.95 decimal digits. Anything beyond that will lose precision. — Some programmer dude, Feb 14 '19 at 15:57
first off I think the expectation is broken of the results. not the code/compiler itself. but start by examining the disassembly to see that both/all start with the same conversion to binary of those constants... — old_timer, Feb 14 '19 at 15:58
And yes, [replacing `-Ofast` by `-O2` on coliru](http://coliru.stacked-crooked.com/a/c39d17e908153635) turns the 9.368... into the 9.370... — YSC, Feb 14 '19 at 16:00
@markzzz Links can go stale, making the question worthless if those links contained crucial information. Try to make the question self-contained. — Some programmer dude, Feb 14 '19 at 16:01
@markzzz The flags should be *in* the question, not behind external links. As far as I'm concerned, if I have to follow a link, it doesn't exist as part of the question. — Jesper Juhl, Feb 14 '19 at 16:02

Max Langhof · Accepted Answer · 2019-02-14T16:12:24.310

6

Even without -Ofast, the C++ standard does not require implementations to be exact with log (or sin, or exp, etc.), only that they be within a few ulp (i.e. there may be some inaccuracies in the last binary places). This allows faster hardware (or software) approximations, which each platform/compiler may do differently.

(The only floating point math function that you will always get perfect results from on all platforms is sqrt.)

More annoyingly, you may even get different results between compilation (the compiler may use some internal library to be as precise as float/double allows for constant expressions) and runtime (e.g. hardware-supported approximations).

If you want log to give the exact same result across platforms and compilers, you will have to implement it yourself using only +, -, *, / and sqrt (or find a library with this guarantee). And avoid a whole host of pitfalls along the way.

If you need floating point determinism in general, I strongly recommend reading this article to understand how big of a problem you have ahead of you: https://randomascii.wordpress.com/2013/07/16/floating-point-determinism/

edited Feb 14 '19 at 16:12

answered Feb 14 '19 at 16:03

Max Langhof

23,383
5
39
72

There's also some obscure flag to disallow the compiler to perform floating point operation on the hardware natural float size (generally 80 bits). I've read something somewhere. – YSC Feb 14 '19 at 16:06
1

This is also worth a read: [What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) – Jesper Juhl Feb 14 '19 at 16:07
@YSC See https://en.cppreference.com/w/cpp/types/climits/FLT_EVAL_METHOD. – Max Langhof Feb 14 '19 at 16:11
@MaxLanghof: the real "deal" is that if I remove the ln2per12 as const, the results are the same. It seems that `const` do somethings different... http://coliru.stacked-crooked.com/a/d246fa05447efe19 – markzzz Feb 14 '19 at 16:17
1

@markzzz See the paragraph about differences between compilation and run time. (Also see the linked article under "Transcendentals".) With `const`, the compiler can constant-fold the expression and replace it with a precise value at compile-time. Without, it has to initialize it at runtime, which may use a completely different method for calculating `log`. – Max Langhof Feb 14 '19 at 16:21
@ Max Langhof: if that's true, I would expect to see different outputs for that value when I `cout` it. Instead, its the same. Try this: http://coliru.stacked-crooked.com/a/c113f402cf81e667 . It hasn't `log` at all, `b` is printed the same (with/without `const`), but `c / b` lead to different results with/without set `const` on `b` – markzzz Feb 15 '19 at 09:47
@markzzz Looking at the assembly [with](https://godbolt.org/z/4u-cgr) and [without const](https://godbolt.org/z/SqfmeD), the compiler constant-folds everything in both cases, no calculation happens at runtime. The difference is surprising, I agree, but fully within the range of conceivable compiler optimizations. For the record, if you use `-O3` instead, you get the same result in both cases. If you want to know where precisely the difference comes from you'll have to dig through `gcc` optimization internals, but you have to live with it regardless. – Max Langhof Feb 15 '19 at 09:54
Its not clear at all why this is happens. Since its not related with Log anymore, I think I would delete this question and open a "more one" with a precise target. Can you remove my question? Since you have replied, I can't... – markzzz Feb 15 '19 at 09:57
What's the point of removing the question? Just ask a new one if you want to, but I don't think you'll get a good answer to "why does `gcc` optimize this in different ways". If you're lucky someone will guide you through the optimization sources, but what would you gain from it? – Max Langhof Feb 15 '19 at 10:04
Let see what's happening! – markzzz Feb 15 '19 at 10:08

Why this same code produce two different fp results on different Machines?

1 Answers1

Linked