1

I'm testing some bits of code, a number which involves computation using floating-point values - often very large numbers of these. I have some generic (C++-templated, but it doesn't really matter for the sake of this question) code which compares my outputs, be they scalar or arrays, against their expected values.

I'm faced with the problem of choosing a precision threshold, at least for the two C/C++ floating-point types float and double - for various functions I'm testing. As is well known, there is no one-size-fits-all with respect to comparing floating-point values, nor a single precision value which fits and computation based solely on the data type: Relative vs. absolute error, numerous operations which may magnify floating-point rounding errors a lot, computations which are supposed to arrive at 0 so you can't really normalize by the expected value, etc.

What is a generally-reasonable approach/algorith/rule-of-thumb to choosing a comparsion method (and equality thresholds) for floating point values?

Community
  • 1
  • 1
einpoklum
  • 118,144
  • 57
  • 340
  • 684

2 Answers2

1

I like the approach used in googletest, e.g. EXPECT_DOUBLE_EQ(a,b) and EXPECT_FLOAT_EQ(a,b): the numbers are approximately equal if they are within 4 units in the last position (4 ULP). To do this, you

  1. convert signed-magnitude to offset
  2. subtract as though they were integers
  3. check that the difference <= 4.

This automatically scales for magnitude and relaxes to absolute near zero.

Reality Pixels
  • 376
  • 2
  • 6
0

There is no generally-reasonable approach :-(

One important property of numbers is that the set of numbers can be divided into equivalence classes where all members of the same equivalence class are "equal" in some sense and all members of two different equivalence classes are "not equal". That property is essential for sorting algorithms and hashing.

If you take double with 53 bit mantissa, and just replace the last bits of the mantissa with zeroes, then you still have equivalence classes, and sorting / hashing will work just fine. On the other hand, two numbers can be arbitrarily close together and still compare equal with this method.

The other method is having an algorithm that decides if two numbers are "possibly equal". You can base everything else on this. For example, a is "definitely greater" than b if a > b and a is not "possibly equal" to b. a is "possibly greater" than b if a > b or a is "possibly equal" to b.

Sorting is problematic. You could have a "possibly equal" to b, and b "possibly equal" to c, but a is not "possibly equal" to c.

If you use double with 53 bits mantissa, then it is unlikely that two unrelated numbers are equal within even 45 bits. So you could check quite reasonably whether the absolute value of the difference is less than the absolute value of the larger number, divided by 2^45. Your mileage will vary considerably. Important is whether you think 0 should be equal to very small numbers or not.

gnasher729
  • 51,477
  • 5
  • 75
  • 98