-1

I'm seeing an odd issue in my code

In my mfc/c++ application, there is a function that gets called at 2 different points. Once, when a dialog/component is displayed. The second time is when some generation is happening from a 2nd dialog/component. The only difference is that from the 2nd dialog, the call is happening in a new thread.

Within this function, broken down to its simplest, 2 doubles are subtracted.

double a = -13.497999999999999
double b = 33.564999999999998 
a - b

when I step through the code, I am getting different results from the subtraction and is having a knock on affect to other calculations. The result from Dialog 2 seems to be closer to what is valid.

Dialog 1 - 20.066999435424805 
Dialog 2 - 20.067000000000000

Very odd that simple subtraction would return different results. The differences may be minimal, but they do build up to cause a larger problem.

using VS2010, settings all correct as far as I can tell.

I've tried using floats and long double's to see if that would resolve anything but they don‘t.

Yu Hao
  • 119,891
  • 44
  • 235
  • 294
mjc
  • 1
  • 1

1 Answers1

1

According to this URL a double holds 15-16 decimal digits:

Double precision - decimal places

12 345678901234567 The value -13.497999999999999 has 17 digits.

The difference between the subtract values is only 0.000000564575195; this difference is within the 16 decimal digit level of precision of a double..

1 234567890123456 0.000000564575195

From this URL also:

http://en.wikipedia.org/wiki/Double-precision_floating-point_format

There are 53 binary bits reserved for the digits in the number; this gives 15 to 17 decimal digits of precision

Community
  • 1
  • 1
A B
  • 4,068
  • 1
  • 20
  • 23