1

I am using fixed decimal point number (using uint16_t) to store percentage with 2 fractional digits. I have found that the way I am casting the double value to integer makes a difference in the resulting value.

const char* testString = "99.85";
double percent = atof(testString);
double hundred = 100;
uint16_t reInt1 = (uint16_t)(hundred * percent);
double stagedDouble = hundred * percent;
uint16_t reInt2 = (uint16_t)stagedDouble;

Example output:

percent:      99.850000 
stagedDouble: 9985.000000
reInt1:       9984
reInt2:       9985

The error is visible in about 47% of all values between 0 and 10000 (of the fixed point representation). It does not appear at all when casting with stagedDouble. And I do not understand why the two integers are different. I am using GCC 6.3.0.

Edit: Improved code snippet to demonstrate percent variable and to unify the coefficient between the two statements. The change of 100 into a double seems as a quality change that might affect the output, but it does not change a thing in my program.

Mike Kinghan
  • 55,740
  • 12
  • 153
  • 182
Krzych
  • 83
  • 1
  • 6
  • 4
    Casting to integer is truncating the fractional part. I guess you can take it from here. – Eugene Sh. May 16 '19 at 16:51
  • 3
    are you compiling in 32-bit mode? Then it's due to the higher precision of x87 registers. There are a lot of duplicates: [C fundamentals: double variable not equal to double expression?](https://stackoverflow.com/q/37626687/995714), [Apparently identical math expressions with different output](https://stackoverflow.com/q/33795031/995714), [Difference in casting float to int, 32-bit C](https://stackoverflow.com/q/54881541/995714), [Difference in floating point arithmetics between x86 and x64](https://stackoverflow.com/q/22710272/995714) – phuclv May 16 '19 at 16:58
  • or maybe `percent` is a `float`? then it's a different issue – phuclv May 16 '19 at 16:59
  • @ryyker This is not the case here. It is not about reinterpreting the bit-pattern. – Eugene Sh. May 16 '19 at 17:04
  • 1
    How have you defined `percent` ? – pablo1977 May 16 '19 at 17:18
  • Are you sure it is 47%? I measure 42.33%. – Eric Postpischil May 16 '19 at 17:27
  • @EugeneSh.: The conversion to integer explains why some results give values less than one might expect. It does not explain why the two methods, both of which convert to the same integer type, differ. – Eric Postpischil May 16 '19 at 17:50
  • 3
    @phuclv, your comment with links to similar problems seems to explain my issue. Thanks! – Krzych May 17 '19 at 08:41
  • Possible duplicate of [Apparently identical math expressions with different output](https://stackoverflow.com/questions/33795031/apparently-identical-math-expressions-with-different-output) – phuclv May 18 '19 at 09:13

2 Answers2

2

Is percent a float? If so, look at what types you're multiplying.

reInt1 is double * float and stagedDouble is int * float. Mixing up floating point math can cause these types of rounding errors.

Changing the 100's to be both double or both int results in the same answer.

cj.vaughter
  • 248
  • 2
  • 6
  • percent is not a float, it is another double. Changing 100 to be in both uniform in both statements `100` or `100.0` does not change anything for me. – Krzych May 17 '19 at 08:21
2

The reported behavior is consistent with percent being declared float, and the use of IEEE-754 basic 32-bit and 64-bit binary floating-point for float and double.

uint16_t reInt1 = (uint16_t)(100.0 * percent);

Since 100.0 is a double constant, this converts percent to double, performs a multiplication in double, and converts the result to uint16_t. The multiplication may have a very slight rounding error, up to ½ ULP of the double format, a relative error around 2−53.

double stagedDouble = 100 * percent;
uint16_t reInt2 = (uint16_t)stagedDouble;

Since 100 is an int constant, this converts 100 to float, performs a multiplication in float, and converts the result to uint16_t. The rounding error in the multiplication may be up to ½ ULP of the float format, a relative error around 2−24.

Since all of the values are near hundredths of an integer, a 50:50 ratio of errors up:down would make about half the results just under what is needed for the integer threshold. In the multiplications, all those with values that are 0, 25, 50, or 100 one-hundredths would be exact (because 25/100 is ¼, which is exactly representable in binary floating-point), so 96/100 would have rounding errors. If the directions of the float and double rounding errors behave as independent, uniform random variables, about half would round in different directions, producing different results, giving about 48% mismatches, which is consistent with the 47% reported in the question.

(However, when I measure the actual results, I get 42% differences between the float and double methods. I suspect that has something to do with the trailing bits in the float multiplication before rounding—the distribution might not act like a uniform distribution of two possibilities. It may be the OP’s code prepares the percent values in some way other than dividing an integer value by 100.)

Eric Postpischil
  • 195,579
  • 13
  • 168
  • 312
  • After the code in question has been changed to use double everywhere (instead of float), the answer becomes irrelevant. The real answer is in comments to the question. – Alex Che Nov 13 '19 at 08:29