Floating Point, how much can I trust less than / greater than comparisons?

Question

Let's say I have two floating point numbers, and I want to compare them. If one is greater than the other, the program should take one fork. If the opposite is true, it should take another path. And it should do the same thing, if the value being compared is nudged very slightly in a direction that should still make it compare true.

It's a difficult question to phrase, so I wrote this to demonstrate it -

float a = random();
float b = random();  // always returns a number (no infinity or NaNs)

if(a < b){
    if( !(a < b + FLOAT_EPISILON) ) launchTheMissiles();
    buildHospitals();

}else if(a >= b){
    if( !(a >= b - FLOAT_EPISILON) ) launchTheMissiles();
    buildOrphanages();

}else{
    launchTheMissiles();  // This should never be called, in any branch
}

Given this code, is launchTheMissiles() guaranteed to never be called?

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html — Ed S., Mar 12 '13 at 23:48
@Clairvoire: I see no guarantee that `b + FLOAT_EPISILON` isn't infinity, which may cause missiles. — Mooing Duck, Mar 12 '13 at 23:49
What is `FLOAT_EPISILON`? Do you mean `FLT_EPSILON`, defined in `` or ``? — Keith Thompson, Mar 12 '13 at 23:52
@JerryCoffin: He mentions a guarantee that `a` and `b` are both not NaN and not Infinity or -Inf. — Mooing Duck, Mar 12 '13 at 23:53
It's poor style, and sometimes dangerous, to compare boolean values for equality to `true` or `false`. `x > y` is already a condition. If you want to negate it, use the `!` operator. — Keith Thompson, Mar 12 '13 at 23:53
If `b` is so close to the maximum representable float that `+ FLOAT_EPISILON` will cause it to be larger than it and overflow, it will become `Infinity` and that will cause problems. Similar for negative infinity. Otherwise you are fine. — Patashu, Mar 12 '13 at 23:56
@Patashu: Assuming that's supposed to be `FLT_EPSILON`, adding that value to a very large number will probably yield the same number. I don't think it's guaranteed, though; in theory, it *could* overflow. (And I don't think there's a guarantee that an overflow yields Infinity.) — Keith Thompson, Mar 13 '13 at 00:01
@Keith Thompson You're right, it's hard to overflow by addition :) >>> sys.float_info.max+sys.float_info.max/100000000000000000 1.7976931348623157e+308 >>> sys.float_info.max+sys.float_info.max/10000000000000000 inf — Patashu, Mar 13 '13 at 00:06
You may have been deceived by the *"always compare floats with epsilon"*-rubbish going around, but no, `launchTheMissles` can never ever be called (well, except for `NaN`s). — Christian Rau, Mar 13 '13 at 08:29

liori · Accepted Answer · 2013-03-13T00:07:34.650

13

If you can guarantee that a and b are not NaNs or infinities, then you can just do:

if (a<b) {
    …
} else {
    …
}

The set of all floating point values except for infinities and NaNs comprise a total ordering (with a glitch with two representations of zero, but that shouldn't matter for you), which is not unlike working with normal set of integers — the only difference is that the magnitude of intervals between subsequent values is not constant, like it is with integers.

In fact, the IEEE 754 has been designed so that comparisons of non-NaN non-infinity values of the same sign can be done with the same operations as normal integers (again, with a glitch with zero). So, in this specific case, you can think of these numbers as of “better integers”.

edited Mar 13 '13 at 00:07

answered Mar 12 '13 at 23:56

liori

40,917
13
78
105

Can you explain the 'glitch with zero'? Does it cause any 'gotchas'? – Patashu Mar 13 '13 at 00:00
Zero has two possible representations in floating point notation. Both of them set significand and exponent fields to zero; one of them sets the sign to positive, the other to negative. Both of these encodings results in the same number, mathematically. The simplest thing you can do to keep things working is just to detect one of these cases and convert it to the other one during the comparison. The FPU will do that for you. – liori Mar 13 '13 at 00:04

score 5 · Answer 2 · answered Mar 12 '13 at 23:50

5

Short answer, it is guaranteed never to be called.

If a<b then a will always be less than b plus a positive amount, however small. In which case, testing if a is less than b + an amount will be true.

The third case won't get reached.

answered Mar 12 '13 at 23:50

Chris K

435
2
10

I accidentally downvoted you. Can you edit the post to remove the vote block ? – Thorsten S. Mar 13 '13 at 15:25

masrtis · Answer 3 · 2013-03-13T17:39:49.410

2

The IEEE 754 (floating point) standard states that addition or subtraction can result in a positive or negative infinity, so b + FLOAT_EPSILON and b - FLOAT_EPSILON can result in positive or negative infinity if b is FLT_MAX or -FLT_MAX. The floating point standard also states that infinity compares as you would expect, with FLT_MAX < +infinity returning true and -FLT_MAX > -infinity.

For a closer look at the floating point format and precision issues from a practical standpoint, I recommend taking a look at Christer Ericson's book Real Time Collision Detection or Bruce Dawson's blog posts on the subject, the latest of which (with a nice table of contents!) is at http://randomascii.wordpress.com/2013/02/07/float-precision-revisited-nine-digit-float-portability/.

edited Mar 13 '13 at 17:39

answered Mar 13 '13 at 00:08

masrtis

1,282
17
32

I accidentally downvoted you. Can you edit the post to remove the vote block ? – Thorsten S. Mar 13 '13 at 15:24
It's worthwhile to note that when if `x` and `y` are real numeric quantities, `xf` is the best `float` representation of `x`, and `yd` is the best `double` representation of `y`, `xf > (float)yd` will only be true if `x` is either greater than `y` or *very* close to it (within half an LSB of a *`double`*), but `(double)xf > yd` may be true even if `y` is hundreds of orders of magnitude greater than `x`. – supercat Jun 13 '13 at 22:03

Pete Becker · Answer 4 · 2013-03-13T15:54:32.953

2

Tests for inequality are exact, as are tests for equality. People get confused because they don't realize that the values they are working with might not be exactly what they think they are. So, yes, the comment on the final function call is correct. That branch will never be taken.

edited Mar 13 '13 at 15:54

answered Mar 13 '13 at 01:08

Pete Becker

74,985
8
76
165

I accidentally downvoted you. Can you edit the post to remove the vote block ? – Thorsten S. Mar 13 '13 at 15:25
I thought tests for equality on floating point values are not exact? Isn't that the point of the EPSILON defines for floats and doubles? – BackDoorNoBaby Nov 04 '15 at 17:15
My understand was that if you want to compare float a and float b for equality, it would have to be something like if (fabs(a - b) > FLT_EPSILON) then your values are not the same – BackDoorNoBaby Nov 04 '15 at 17:19
1

@BackDoorNoBaby - if two floating-point values have the same value, they are equal. If they do not have the same value they are not equal. If your code computes the "same" value in two different ways it may end up with results that are not equal. Some people like to test whether they are "nearly" equal by checking with tolerances. But that treats the symptom, not the underlying problem. `FLT_EPSILON` has nothing to do with this; it's simply the smallest value that can be added to `1.0f` to produce a different value, i.e., it gives you information about the underlying representation. – Pete Becker Nov 04 '15 at 18:26
@PeteBecker - perhaps there is a misunderstanding: If I am writing a function to compare two floating point values for equality, I cannot simply use the '==' operator. I would need to check if the absolute value of the difference between the two numbers is greater than the FLT_EPSILON value. If the difference is indeed greater, I know the floating point values are not the equal. Correct? – BackDoorNoBaby Nov 05 '15 at 18:37
@BackDoorNoBaby - see [this thread](http://stackoverflow.com/questions/19754949/what-is-a-standard-way-to-compare-float-with-zero/19755324#19755324). – Pete Becker Nov 05 '15 at 19:47
@PeteBecker I agree with the thread in terms of comparing a floating point variable to a floating point value constant, but perhaps I was confused because the OP was inquiring about comparing two floating point variables – BackDoorNoBaby Nov 05 '15 at 20:40

score 1 · Answer 5 · answered Sep 20 '17 at 16:14

What about less than check with an epsilon window ? if a is less than b then a can not be equal to b

/**
 * checks whether a <= b with epsilon window
 */
template <typename T>
bool eq(T a, T b){
    T e = std::numeric_limits<T>::epsilon();
    return std::fabs(a-b) <= e;
}
/**
 * checks whether a < b with epsilon window
 */
template <typename T>
bool lt(T a, T b){
     if(!eq(a,b)){ // if a < b then a != b
         return a < b;
     }
     return false;
}
/**
 * checks whether a <= b with epsilon window
 */
template <typename T>
bool lte(T a, T b){
     if(eq(a,b)){
         return true;
     }
     return a < b;
}

Floating Point, how much can I trust less than / greater than comparisons?

5 Answers5