Why does the sum of float numbers with exact values still depend on order?

Question

Suppose I have 3 float numbers, which can be represented by float number accurately:

var a=0.1000000000000000055511151231257827021181583404541015625;
var b=0.200000000000000011102230246251565404236316680908203125;
var c=0.299999999999999988897769753748434595763683319091796875;

they are the actual value of 0.1,0.2,and 0.3 stored in float. I think a+b+c should be equals to c+b+a, because both a, b and c are already rounded and is the exact value, their sum should not depend on order, but now I tested it is not:

var a=0.1000000000000000055511151231257827021181583404541015625;
var b=0.200000000000000011102230246251565404236316680908203125;
var c=0.299999999999999988897769753748434595763683319091796875;
console.log(a+b+c==c+b+a);

What is the reason?

Note: I'm not asking about Is floating point math broken?, because 0.1000000000000000055511151231257827021181583404541015625 is already the accurate value in floating point number (some float numbers can be represented without rounding errors, for example 0.5,0.75 and 0.375, see:Is hardcode float precise if it can be represented by binary format in IEEE 754?).

My assumption is : a, b and c have no rounding errors, so their sum should also have no rounding errors at all, just like

1+2+3 == 3+2+1
0.5+0.375+0.25 == 0.25+0.375+0.5

but now a+b+c isn't in the case, what's wrong of my assumption here?

Answer 1

Sums Are Not Exact

Your assumption that the sum of exact values is exact is wrong.

Floating-point arithmetic uses some number of digits that is fixed for the format (such as 24 binary digits for float). The mathematical sum of two 24-digit numbers may have 25 digits and therefore requires rounding to represent within 24 digits (and an exponent).

Additionally, when two numbers with different exponents are added, the digits of one are offset relative to the other. The sum may have additional digits due to the offset, and again must be rounded.

When you add the numbers in different orders, the resulting roundings may be different.

Examples of Inexact Sums

These examples use three-digit binary significands.

In this example, the addition carries into a new column:

 1.10 • 23
 1.01 • 23
――――――――――
10.11 • 23 Exact sum, too many digits, must be rounded.
11.0  • 23 Sum rounded to three digits.
1.10  • 24 Rounded sum, exponent adjusted to normalize significand.

In this example, the numbers have different exponents, and adjusting for this shifts the digits into new columns:

 1.11 • 23
 1.01 • 25   Different exponent requires adjustment.

 0.0111 • 25 Adjusted to match exponent.
 1.01   • 25
――――――――――――
 1.1011 • 25 Exact sum, too many digits, must be rounded.
 1.11   • 25 Rounded sum.

Examples of Non-Associative Sums

Now we can look at adding three numbers in different ways and see that different sums are produced.

We will compare (1.10•2⁰ + 1.10•2⁰) + 1.00•2⁴) to 1.10•2⁰ + (1.10•2⁰ + 1.00•2⁴).

For the first express, we add the first and second operands, then the third:

Add first and second operands:
 1.10 • 20 First operand.
 1.10 • 20 Second operand.
――――――――――
11.00 • 20 Exact sum, too many digits, must be rounded.
11.0  • 20 Rounded sum, must be normalized.
1.10  • 21 Normalized, rounded sum.

Add previous result and third operand:
 1.10 • 21 Previous result.
 1.00 • 24 Third operand.

Exponents do not match, so adjust and then add:
 0.00110 • 24 Previous result adjusted to match exponent.
 1.00    • 24 Third operand.
――――――――――――
 1.00110 • 24 Exact sum, too many digits, must be rounded.
 1.01    • 24 Rounded sum.

For the second expression, we add the second and third operands, then the first:

Add second and third:
 1.10    • 20 Second operand.
 1.00    • 24 Third operand.

Exponents do not match, so adjust, then add:
 0.000110 • 24 Second operand adjusted to match exponent.
 1.00     • 24 Third operand.
――――――――――――――
 1.000110 • 24 Exact sum, too many digits, must be rounded.
 1.00     • 24 Rounded sum.

Add first operand and previous result:
 1.10     • 20 First operand.
 1.00     • 24 Previous result.

Exponents do not match, so adjust and then add:
 0.000110 • 24 First operand adjusted to match exponent.
 1.00     • 24 Previous result.
―――――――――――――
 1.000110 • 24 Exact sum, too many digits, must be rounded.
 1.00     • 24 Rounded sum.

The first expression yields 1.01•2⁴, while the second expression yields 1.00•2⁴. So the order in which operands are added affects the result.

Answer 2

My assumption would be that the intermediate values of the addition may have different rounding point errors. For example, a + b might incur rounding error. And b + c might not or the resulting value might cause a different type of float error when added with the final value. Adding parentheses makes the order of operations and intermediate values more clear:

(a + b) + c

(c + b) + a