16

I stumbled upon a difference in the way floating point arithmetics are done between MS VS 2010 builds for x86 and x64 (both executed on the same 64 bit machine).

This is a reduced code sample:

float a = 50.0f;
float b = 65.0f;
float c =  1.3f;
float d = a*c;
bool bLarger1 = d<b;
bool bLarger2 = (a*c)<b;

The boolean bLarger1 is always false (d is set to 65.0 in both builds). Variable bLarger2 is false for x64 but true for x86!

I am well aware of floating point arithmetics and the rounding effects taking place. I also know that 32 bit sometimes uses different instructions for floating operations than 64 bit builds. But in this case I am missing some information.

Why is there a discrepency between bLarger1 and bLarger2 on the first place? Why is it only present on the 32 bit build?

Left: x86, Right: x64

Oliver Zendel
  • 2,695
  • 34
  • 29
  • 1
    my guess is that the x86 version is using the FPU registers for that and x64 is using SSE registers for that. But you probably need to look at IL code and also at the machine code. – Lev Mar 28 '14 at 10:37
  • 1
    Certainly x86 is using x87 unit, and x64 is using SSE unit. But it doesn't really explain the difference. They should both get the same answer. @Oliver can you show how you are compiling the code because my quite attempt at a repro failed. Both bools are `false` for both x86 and x64 for me. – David Heffernan Mar 28 '14 at 10:39
  • OK, now I got a repro! – David Heffernan Mar 28 '14 at 10:46
  • 1
    It's going to be the way a*c is treated in the expression for `bLarger2`. I guess it will be a float mult in one and a double mult in the other, or something like that – David Heffernan Mar 28 '14 at 10:49
  • 1
    The real question is why is this surprising, knowing full well that floating point isn't exact. Depending on compiler, compiler options, etc. results can differ. – PaulMcKenzie Mar 28 '14 at 10:53
  • Well the difference between two comparision operations on the same platform that look the same was especially confusing for me – Oliver Zendel Mar 28 '14 at 11:19
  • 1
    @PaulMcKenzie Whilst floating point arithmetic does not exactly represent all real values, it is repeatable and well-defined. It is not unreasonable to hope for consistency between different compilers. – David Heffernan Mar 28 '14 at 11:29
  • Related questions: [Does any floating point-intensive code produce bit-exact results in any x86-based architecture?](https://stackoverflow.com/questions/27149894/does-any-floating-point-intensive-code-produce-bit-exact-results-in-any-x86-base) and [this one](https://stackoverflow.com/questions/33789476/is-boostrandomuniform-real-distribution-supposed-to-be-the-same-across-proce) – Gabriel Devillers May 24 '22 at 08:14

2 Answers2

23

The issue hinges on this expression:

bool bLarger2 = (a*c)<b;

I looked at the code generated under VS2008, not having VS2010 to hand. For 64 bit the code is:

000000013FD51100  movss       xmm1,dword ptr [a] 
000000013FD51106  mulss       xmm1,dword ptr [c] 
000000013FD5110C  movss       xmm0,dword ptr [b] 
000000013FD51112  comiss      xmm0,xmm1

For 32 bit the code is:

00FC14DC  fld         dword ptr [a] 
00FC14DF  fmul        dword ptr [c] 
00FC14E2  fld         dword ptr [b] 
00FC14E5  fcompp

So under 32 bit the calculation is performed in the x87 unit, and under 64 bit it is performed by the x64 unit.

And the difference here is that the x87 operations are all performed to higher than single precision. By default the calculations are performed to double precision. On the other hand the SSE unit operations are pure single precision calculations.

You can persuade the 32 bit unit to perform all calculations to single precision accuracy like this:

_controlfp(_PC_24, _MCW_PC);

When you add that to your 32 bit program you will find that the booleans are both set to false.

There is a fundamental difference in the way that the x87 and SSE floating point units work. The x87 unit uses the same instructions for both single and double precision types. Data is loaded into registers in the x87 FPU stack, and those registers are always 10 byte Intel extended. You can control the precision using the floating point control word. But the instructions that the compiler writes are ignorant of that state.

On the other hand, the SSE unit uses different instructions for operations on single and double precision. Which means that the compiler can emit code that is in full control of the precision of the calculation.

So, the x87 unit is the bad guy here. You can maybe try to persuade your compiler to emit SSE instructions even for 32 bit targets. And certainly when I compiled your code under VS2013 I found that both 32 and 64 bit targets emitted SSE instructions.

David Heffernan
  • 601,492
  • 42
  • 1,072
  • 1,490
  • Interesting, so I assume the difference between the way bLarger1 and bLarger2 are computed is the precision of the ALU. For bLarger2 the result with the higher precision is kept in the ALU for comparision while bLarger1 will first load the single precision values and compare those. – Oliver Zendel Mar 28 '14 at 11:24
  • @OliverZendel Correct – David Heffernan Mar 28 '14 at 11:28
  • Hm, neither changing "Enable Enhanced Instruction Set" nor "Floating Point Model" resulted in a different behaviour (VS2010 x86); _controlfp does the trick, but I'm still curious if there is a compiler setting itself that results in the same behaviour – Oliver Zendel Mar 28 '14 at 11:48
  • 3
    @OliverZendel There's no compiler setting that will change the control word. That is a runtime property and so has to be managed at runtime. And it's a total minefield FWIW with modules left right and centre screwing with the control word. Compile with `/arch:SSE2` and you'll avoid all that x87 pain. But your program will only run on machines with SSE2 units. – David Heffernan Mar 28 '14 at 12:00
  • That's a limitation I can live with (in this case). Thanks! – Oliver Zendel Mar 28 '14 at 12:14
  • 1
    Actually I'm not even sure that /arch:SSE2 will necessarily get it done. Anyway, I think we do at least understand the cause for the behaviour. – David Heffernan Mar 28 '14 at 12:18
  • I tried it out and in combination with "Fast" Floating Point Model (/fp:fast) the sample behaviour is consistent (both booleans are false on both platforms) – Oliver Zendel Mar 28 '14 at 12:23
  • Excellent. You program will be a little faster too with SSE2. – David Heffernan Mar 28 '14 at 12:30
  • Note that the solution references _controlfp(_MCW_PC, _PC_24); however, the Microsoft documentation and Intellisense in VS2017 indicates that the mask is the *second* parameter. https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/control87-controlfp-control87-2 Which means, that this line should read: _controlfp(_PC_24, MCW_PC) Here are the docs for 2008, which state the same. https://msdn.microsoft.com/en-us/library/e9b52ceh(v=vs.90).aspx – Markus Mar 01 '18 at 02:17
  • @Markus You are correct. I have changed the answer accordingly. Thanks. – David Heffernan Mar 01 '18 at 07:12
-2

Floating points operations are always imprecise, and comparing two floats this close (or equal) almost never return the correct output.

Floating point numbers are stored and processed differently on 32bit and 64bit machines (as also suggested by comments). If I remember correctly, in VC 32bit floats are saved on the stack and FPU (Floating-Point Unit) processes them, whereas floats on a 64bit machine can be stored in specialized registers (SSE) and are calculated using other Units in the CPU.

I have no definite source to my answer, but please look at this page or this.

Janman
  • 1,030
  • 2
  • 12
  • 20
  • 1
    There is no difference in storage. These are IEEE754 floats. A standard format. – David Heffernan Mar 28 '14 at 11:11
  • @DavidHeffernan, They are stored the same way, but different locations is what I'm trying to say. – Janman Mar 28 '14 at 11:30
  • 1
    But floats are not necessarily stored on the stack. The x87 unit has its own register stack of 8 extended precision floating point registers ST(0) to ST(7). They are specialized registers for sure. And the SSE unit has its own specialized registers. – David Heffernan Mar 28 '14 at 11:40