Double's multiplication is less precise than float's one

Question

Suppose we have the equation y = k1 * x + b1 = k2 * x + b2. Let's calculate x in floats. I know that it's a bad choice, but I want to understand the reason for results I get. Also let's calculate y using this x and then do the same, but with double(x). Consider this code:

std::cout.precision(20);

float k1, b1, k2, b2;
std::cin >> k1 >> b1 >> k2 >> b2;

float x_f = (b2 - b1) / (k1 - k2);
double x_d = x_f;
printFloat(x_f); // my function which prints number and it's binary representation
printDouble(x_d);
float y_f = x_f * k1 + b1;
double y_d = x_d * k1 + b1;
printFloat(y_f);
printDouble(y_d);

And with k1 = -4653, b1 = 9968, k2 = 520, b2 = -1370 surprisingly got the following results:

x_f = 2.19176483154296875 01000000000011000100010111100000
x_d = 2.19176483154296875 0100000000000001100010001011110000000000000000000000000000000000
y_f = -230.2822265625 11000011011001100100100001000000
y_d = -230.28176116943359375 1100000001101100110010010000010000110000000000000000000000000000

Whereas more precise answer (calculated with Python Decimal) is:

x = 2.191764933307558476705973323023390682389
y = -230.28223468006959211289387202783684516

And the float's answer is closer than the double's one! Why is this happening? I've debugged with gdb (compiled on 64-bit Ubuntu 14.04 g++ 4.8.4) and viewed the instructions, and they are all ok, so it's due to multiplication.

The "more precise answer" exactly matches what Gnome Calculator gives (probably not a coincidence). @n.m. , I think you better explain why it's wrong. — ams, Aug 19 '15 at 19:52
I can't work out why the float/double numbers are different at all? Converting float to double ought to be harmless. — ams, Aug 19 '15 at 19:53
Maybe this article about [What Every Computer Scientist Should Know About Floating Point](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) should sum things up (and close this question). — Thomas Matthews, Aug 19 '15 at 20:03
@Dieter, I don't understand what flaw do you mean. I've just convert float to double and got worse result. — perlik, Aug 19 '15 at 20:04
@perlik - x_d should be calculated using double precision: double x_d = (double)(b2 - b1) / (double)(k1 - k2); With this change for x_d and y_d I get 2.1917649333075584 -230.28223468006945 . — rcgldr, Aug 19 '15 at 20:10
@rcgldr, yes it's a true. But when I use x_f and calculate float's y_f, why I get better result than with x_d (wich is the same as x_f, because double is wider) and y_d = x_d * k1 (which also casts to (double)k1 and it equals to (float)k1). I mean that double is at least as good as float. — perlik, Aug 19 '15 at 20:18
@perlik It's probably due to the rounding errors being in opposite directions, and so happen to cancel each other out. — Simon Byrne, Aug 19 '15 at 20:59
@ThomasMatthews: Do you have a specific passage in that 100-page document you're trying to reference, or are you just trying to be a jerk? — tmyklebu, Aug 19 '15 at 21:45

score 3 · Accepted Answer · answered Aug 19 '15 at 21:38

It's a coincidence that the rounding cancels out and ends up being closer with float than with double. The origin of the difference is that x_d * k1 is promoted to double whereas x_f * k1 is evaluated as float.

To provide a simpler example of how this kind of rounding can cause a lower-precision type to produce a more accurate answer, consider two new numeric types called sf2 and sf3, each of which store base-10 numbers, with 2 and 3 significant digits, respectively. Then consider the following calculation:

// Calculate (5 / 4) * 8. Expected result: 10
sf2 x_2 = 5.0 / 4.0; // 1.3
sf2 y_2 = x_2 * 8.0; // 10
sf3 x_3 = x_2; // 1.30
sf3 y_3 = x_3 * 8.0; // 10.4

Note that using the above types, even though all sf2 values are representable in the sf3 type, the sf2 calculation is more accurate. This is because the round-up of 1.25 to 1.3 when calculating x_2 is exactly canceled when rounding 10.4 to 10. But when the second calculation is done using the sf3 type, the initial round-up is persisted but the round-down no longer occurs.

This is an example of the many pitfalls you will encounter when dealing with floating point types.

Double's multiplication is less precise than float's one

1 Answers1