3

I stumbled upon a problem, where the result of the floating point operation depends on whether I subtract a value now (f) or later (f2). Why is that?

int main(int argc, char * argv[])
{   
    const float fResolution = 0.501f;
    const int iDiff = -6; // some values lead to f == f2, like -1, -2, -3, -4 or -5
    float f = iDiff * fResolution - 10.0f;
    float f2 = iDiff * fResolution;
    f2 -= 10.0f;

    printf("Difference %f %f %f: %d\n", f, f2, f - f2, iDiff);
    return 0;
}

Output: Difference -13.006000 -13.006001 0.000001: -6

I guess, it has something to do with floating point precision, but there are only floats here, so what is the difference between f and f2? Or is it maybe a special behavior in Visual Studio 2010?

This is of course a minimal example. In the real world, I extracted the calculation of iDiff * fResolution into a method, and sometimes subtract -10. The output of my program differed after extracting the method because of the above problem.

Disassembly (Debug Mode):

    const float fResolution = 0.501f;
00CC7872  fld         dword ptr [__real@3f004189 (0D4AEACh)]  
00CC7878  fstp        dword ptr [ebp-0F8h]  
    const int iDiff = -6;
00CC787E  mov         dword ptr [ebp-104h],0FFFFFFFAh  
    float f = iDiff * fResolution - 10.0f;
00CC7888  fld         dword ptr [ebp-0F8h]  
00CC788E  fmul        qword ptr [__real@c018000000000000 (0D4AED8h)]  
00CC7894  fsub        qword ptr [__real@4024000000000000 (0D4AE58h)]  
00CC789A  fstp        dword ptr [ebp-110h]  
    float f2 = iDiff * fResolution;
00CC78A0  fld         dword ptr [ebp-0F8h]  
00CC78A6  fmul        qword ptr [__real@c018000000000000 (0D4AED8h)]  
00CC78AC  fstp        dword ptr [ebp-11Ch]  
    f2 = f2 - 10.0f;
00CC78B2  fld         dword ptr [ebp-11Ch]  
00CC78B8  fsub        qword ptr [__real@4024000000000000 (0D4AE58h)]  
00CC78BE  fstp        dword ptr [ebp-11Ch]
Fabian
  • 4,001
  • 4
  • 28
  • 59
  • 2
    Intermediate calculations can be stored in more precise types. When data is stored in a variable all extra precision is lost. Maybe. If compiler optimises away some variables, final value can change again. Fun fact: `sin(x) == sin(x)` might yield `false` (on older GCC compilers). – Revolver_Ocelot Jul 08 '16 at 14:00
  • 1
    It likely has to do with the intermediate storage of the floating-point values. If you're targeting x86-32 and telling it precision is not of utmost importance, the optimizer is probably generating code that leaves intermediate values on the 80-bit FPU stack, rather than spilling it to memory to preserve constant precision. No way to tell for sure without looking at the disassembly. Which is hardly worth doing, relying on floating point computations for precision is a fool's errand. – Cody Gray - on strike Jul 08 '16 at 14:00
  • obligatory plugs: [Is floating point math broken?](http://stackoverflow.com/questions/588004/is-floating-point-math-broken) and [Floating point calculation gives different results with float than with double](http://stackoverflow.com/questions/14890283/floating-point-calculation-gives-different-results-with-float-than-with-double)(how more precision can affect the results) – NathanOliver Jul 08 '16 at 14:12
  • 2
    The result of the instruction at address 00CC788E is kept stored inside the FPU at 64-bit precision. At address 00CC78AC you force it to be stored back to memory, truncating the result to 24-bit precision. Intel thought 36 years ago that this was a desirable feature, armies of programmers since have disagreed. Fwiw, the same thing will happen in the Release build of your program, but now it is no longer under your control. Do update your VS version, FPU usage was retired at VS2012. – Hans Passant Jul 08 '16 at 14:14
  • Relevant reading : https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html – Jesper Juhl Jul 08 '16 at 14:17
  • @Hans Passant: I call your comment an answer to my question. If you convert it, I can mark my question as solved. Maybe you can add a reference where one can look up the assembler instructions, that would be kind. What do you mean by retired? Doesn't this affect performance? – Fabian Jul 11 '16 at 09:16
  • I already have more answers about this miserable FPU behavior than I care to maintain. There is no mainstream C++ compiler left that still generates FPU code so this issue can be forgotten about. Good riddance, it won't be missed. Update your VS version. – Hans Passant Jul 11 '16 at 09:37

0 Answers0