0

I appreciate the comments and references but I am still not understanding how doubles are stored. I do understand how floats work and thought doubles did the same but used another byte for greater precision. In a nut shell here's what doesn't make sense to me:

 double a = 5.01 is stored as 5.01 how is it able to do that? 
 since .1 can't be represented in binary?

This question is NOT a duplicate of the question cited. In the first case .01 is being stored as .0099999999787 and in the second case .01 is being stored as .01000000000000

The comment that println sometime rounds is plausible but calculations are being treated as if the value is indeed .0100000000000

I have the following example that I don't understand:

double a = 6;
double b = 5.99;
double c = .99;

System.out.println((a-b) + "  " + (a-c));

why is a - b = .0099999999999787  (which seems correct)
but    a - c = 5.01 (which seems incorrect)

shouldn't a - c = 5.0099999999787 ?
DCR
  • 14,737
  • 12
  • 52
  • 115
  • 2
    println rounds in some cases – zapl Sep 29 '18 at 15:19
  • 2
    But stil a-b is not wrong – Ryuzaki L Sep 29 '18 at 15:20
  • 1
    Why do you say "seems incorrect"? What were you expecting? – President James K. Polk Sep 29 '18 at 15:23
  • 1
    *"In the first case .01 is being stored as .0099999999787 and in the second case .01 is being stored as .01000000000000"* No, in one case `0.01` is being stored (as nearly as possible), and in the other case, `5.01` is being stored (as nearly as possible). Those are very different things. `double`s don't work in decimal, they work in binary. (That said, I think there's a better dupetarget for this, one about how numbers are *output*.) – T.J. Crowder Sep 29 '18 at 15:25
  • @T.J could you point me to a reference on how doubles work? – DCR Sep 29 '18 at 15:27
  • 1
    https://en.wikipedia.org/wiki/Double-precision_floating-point_format – zapl Sep 29 '18 at 15:27
  • The dup question is such a reference – President James K. Polk Sep 29 '18 at 15:27
  • @DCR - I think the linked dupe does that (again, it may not be the best dupe for this question; I haven't found the one I **know** I've seen). Here's a reference to how the string representation of a double is determined (the string does not necessarily include all the non-zero digits in the fractional portion, only enough to differentiate the number from the next representable number): https://docs.oracle.com/javase/10/docs/api/java/lang/Double.html#toString(double) – T.J. Crowder Sep 29 '18 at 15:28
  • 1
    I've added what I think is the better dupetarget. I left the other one as well (for future readers), but there's an argument for removing it, too. – T.J. Crowder Sep 29 '18 at 15:32
  • If you have a look at https://ideone.com/UFqFEJ you can see that it's actually rounding that is applied inconsistently – zapl Sep 29 '18 at 15:49
  • To see the exact representation of `0.01` you can do `System.out.println(new BigDecimal(0.01));` which prints `0.01000000000000000020816681711721685132943093776702880859375` The value less than this can be printed with `System.out.println(new BigDecimal(Math.nextDown(0.01)));` as `0.0099999999999999984734433411404097569175064563751220703125` – Peter Lawrey Sep 29 '18 at 17:46
  • You can see how many representable values are between 0.01 and your number, you can do `System.out.println( Double.doubleToRawLongBits(0.01) - Double.doubleToRawLongBits(0.0099999999999787));` which prints `12279` – Peter Lawrey Sep 29 '18 at 17:49
  • BTW on my machine, `a-b` is `0.009999999999999787` (an extra `9`) but even this has a high error of 123 representable values off. – Peter Lawrey Sep 29 '18 at 17:51

0 Answers0