What is precision of double/float comparison (relation operator) with integral types?

Question

I understand that it is not recommended to use floating-point in any comparisons. But so far that this operation is allowed, here is the question:

    int x = 90;
    float y = 90.00_001f; 

    if (x < y) {
        System.out.println("it works!"); // gets printed
    }

    // now just add one zero to lessen the precision a bit...

    int x = 90;
    float y = 90.000_001f;

   if (x < y) {
      System.out.println("it works!"); // not printed
   }

The precision is the precision as it would be with two `float` variables, it's a [widening conversion](https://docs.oracle.com/javase/specs/jls/se8/html/jls-5.html#jls-5.1.2). — Boris the Spider, Sep 29 '17 at 21:56
https://stackoverflow.com/questions/13297207/is-it-valid-to-compare-a-double-with-an-int-in-java — Sotirios Delimanolis, Sep 29 '17 at 21:57
https://stackoverflow.com/questions/41653576/is-it-safe-to-compare-a-float-and-an-int-in-java — Sotirios Delimanolis, Sep 29 '17 at 21:58

score 3 · Answer 1 · answered Sep 29 '17 at 22:04

The problem here is not with the comparison to integer, but with the precision of float itself.

You can specify as many decimal places as you like when you specify a float (or double) literal; but this doesn't mean that the precision will be retained when the code is compiled.

For example, all of the following have the same value:

90.f
90.000_001f
90.000000000000000000000000000000000000000000000000000000000‌00000001f

Ideone demo

So the reason the second comparison fails is that 90.000_001f is equal to 90.f; and that's equal to the value of widening 90 to a float.

score 0 · Accepted Answer · answered Sep 29 '17 at 22:05

0

The precision of a IEEE 754 single-precision floating point is between 6 and 9 significant decimal digits (source).

For double-precision you get between 15 and 17 significant decimal digits.

answered Sep 29 '17 at 22:05

lexicore

42,748
17
132
221

aka.nice · Answer 3 · 2017-10-02T08:12:53.940

The other answers are correct, so I won't repeat them here.

What I want to add is another potential surprise about considering == as an equivalence relation, which normally imply associativity:

(a == b) and (b == c) => (a == c)

But if you try this snippet for example in https://repl.it :

class Main {
  public static void main(String[] args) {
    int i=16777217;
    float f=16777216.0f;
    double d=16777216.0;
    if(i == f) {
       System.out.println("i == f");
    }
    if(d == f) {
       System.out.println("d == f");
    }
    if(i == d) {
       System.out.println("i == d");
    }
  }
}

The surprise is that i==f and f==d but not(i==d)...
This is because when writing i==f, an implicit conversion float(i)==f happens, and this conversion potentially loose precision because an int can require up to 31 bits of precision, while a float offers at most 24.

This could have been different, if you look how Lisp, Scheme or recent Squeak/Pharo Smalltalk handles the comparisons, you'll see that they care for exactness...

Extract from http://www.lispworks.com/documentation/lcl50/aug/aug-170.html

In general, when an operation involves both a rational and a floating-point argument, the rational number is first converted to floating-point format, and then the operation is performed. This conversion process is called floating-point contagion. However, for numerical equality comparisons, the arguments are compared using rational arithmetic to ensure transitivity of the equality (or inequality) relation.

What is precision of double/float comparison (relation operator) with integral types?

3 Answers3