Equality of floating point numbers after storing/loading/moving

Question

A coworker and I have a disagreement about what can happen when comparing two floating-point numbers that have not been mathematically operated on. i.e. the numbers may have been moved around memory and/or cpu registers, but no math has been done on them. Maybe they have been put in a list and then removed or other various operations.

My experience has led me to believe that doing non-arithmetic operations on floating-point numbers should never change them or be subject to the same rounding errors as arithmetic operations. My coworker contents that the floating-point processing part of the cpu for some modern architectures is allowed to slightly corrupt the number such that equality checks fail, even when only storing/loading/moving the value.

For example, consider this C code:

float* a = (float*)malloc(sizeof(float));
float* b = (float*)malloc(sizeof(float));
*a = 1.0;
*b = 1.0;
int equal = *a == *b;

Are there any circumstances where equal would not be 1?

Answer 1

What you wrote is equivalent to

float a = 1.0;
float b = 1.0;
int equal = a == b;

(using pointers do not change anything as far as the C standard is concerned). So, for the variable a, 1.0 is interpreted in some evaluation format F (depending on FLT_EVAL_METHOD, see ISO C17 5.2.4.2.2p9), then converted to float and stored in a. Ditto for b. As a general rule, storing/reading values must not change them, unless explicitly stated (for instance, the ISO C17 standard explicitly says in 6.2.6.2p3 that on implementations that support negative integer zeros, a negative zero may become a normal zero when stored).

To answer the question, first consider the conversion of the 1.0 constant (as a string) to F on both lines. ISO C17 says in 6.4.4.2p5: "All floating constants of the same source form shall convert to the same internal format with the same value." So you'll get the same value (in the evaluation format F) in both cases. But note that if you had 1.0 and 1.00 respectively, you might get different values (unlikely, in particular because 1.0 is exactly representable and simple enough, but not forbidden by the C standard).

Then consider the conversion of the obtained value to float. ISO C17 6.3.1.5p1 says: "When a value of real floating type is converted to a real floating type, if the value being converted can be represented exactly in the new type, it is unchanged." This is the case of the value 1 if 1.0 (used in the example) was converted to 1, so that equal will be 1 in this case. But if 1.0 was converted to some other value, not representable in a float, I think that equal could be 0 (the C standard does not require that conversion of some value to some type always yield the same result, and note that in particular, this is not the case when the rounding mode changes).

Answer 2

Your coworker’s belief might have origins in C and C++ rules.

C and C++ allow floating-point expressions to be evaluated using more precision than the nominal types of the operands.

For example, given all float variables, d = a + b - c; may be evaluated using double arithmetic. The rules require that, when the result is stored in d, it be converted back to the nominal type float. However, the intermediate calculations can use double. For example, if a is 2³⁰, b is 1, and c is 2³⁰, this expression would produce 0 if computed using float arithmetic, because adding 2³⁰ and 1 would produce 2³⁰ (because 2³⁰+1 is not representable in the format commonly used for float, so it is rounded to a representable value), and then subtracting 2³⁰ yields 0. However, if double arithmetic is used, then the addition produces 2³⁰+1, and subtracting 2³⁰ yields 1.

This is not dissimilar from the fact that, given all unsigned char variables, an expression is evaluated using int arithmetic and may produce results different than if it were evaluated using only unsigned char arithmetic. For example, with a = 100, b = 3, d = 3 * a / b ; will yield 100 since int arithmetic is used. If unsigned char arithmetic were used, 3 * a would produce 44 (300−256), and dividing by 3 would produce 14. So it is not solely a floating-point issue; it is an issue about how the programming language evaluates expressions.

The C and C++ standards also require that casts round their operands to the target type, as well as assignments.

This license by the standards does not allow implementations to change floating-point values that are merely copied, including by an assignment to the same type that performs no arithmetic operation.

Answer 3

I'm assuming you're referring to IEEE754 binary floating point numbers, as these would be used by most conventional CPUs these days.

In that case, then yes just moving these around would not cause the values to change. When doing operations on the values, it sounds like your intuition might need clarification. Specifically, the result of any operation should be the same as if the FPU performed the operation at full (i.e. arbitrary) precision and then only rounded the result to fit into the appropriate (e.g. 32bit binary float). Hence, for a CPU that implements IEEE754 floats, any rounding that occurs will be deterministic–this is the reason behind the 0.1 + 0.2 != 0.3 statement.

Note that by using C you've complicated this via two mechanisms; 1. C doesn't require IEEE754 semantics; 2. C will automatically convert between float and double values (i.e. IEEE754 32bit and 64bit binary floats on common architectures). These two features can mean that code might not do what a naive reader expects.

As a more complete example, I put the following code into Godbolt's compiler explorer:

#include <math.h>

int explicit_nan(float * restrict a, float * restrict b, float val) {
    *a = val;
    *b = val;
    return isnan(val) ? 1 : *a == *b;
}

int implicit_nan(float * restrict a, float * restrict b, float val) {
    *a = val;
    *b = val;
    return *a == *b;
}

The restrict keyword is needed otherwise Clang compiles the code defensively by assuming it might be called like:

char data[sizeof(float) + 1];
explicit_nan((float*)(data), (float*)(data+1), 1.f);

Which causes the write to b to change the value of a. Presumably you don't care about this case, but the C compiler can't be so lenient.

The explicit_nan code can be seen to compile to code that always returns 1 (by setting EAX), while the implicit_nan code only does the comparison on the passed parameter so that NaNs are handled correctly in a small amount of code.