0

So I have BigDecimal holding value 0.99 and I invoke:

  • Float.toHexString(rationalNumber.floatValue()) I get 0x1.fae148p-1
  • Double.toHexString(rationalNumber.doubleValue()) I get 0x1.fae147ae147aep-1

I'm thinking since we are representing a small number like 0.99 we should get the same hex value regardless. Agree?

Code (failing test):

@Test public void testDelta() {
    BigDecimal rationalNumber = new BigDecimal("0.99").setScale(2,BigDecimal.ROUND_DOWN);

    String hexFromFloat = Float.toHexString(rationalNumber.floatValue());
    String hexFromDouble = Double.toHexString(rationalNumber.doubleValue());

    String hexFromFloatMsg = rationalNumber.floatValue() + " = " + hexFromFloat;
    String hexFromDoubleMsg = rationalNumber.doubleValue() + " = " + hexFromDouble;

    Assert.assertEquals(hexFromFloatMsg + ", " + hexFromDoubleMsg, hexFromDouble, hexFromFloat);
}

Output:

org.junit.ComparisonFailure: 0.99 = 0x1.fae148p-1, 0.99 = 0x1.fae147ae147aep-1 
Expected :0x1.fae147ae147aep-1
Actual   :0x1.fae148p-1
jakstack
  • 2,143
  • 3
  • 20
  • 37

2 Answers2

2

The difference occurs in these two operations:

rationalNumber.floatValue()
rationalNumber.doubleValue()

Each of these converts a BigDecimal value of .99 to floating-point. In hexadecimal floating-point (with a decimal exponent for a power of two), .99 is 0x1.fae147ae147ae147…p-1. When this is converted to float, only 24 bits of the significand (fraction part) can be stored, because that is all the bits the float type has for a significand. (23 bits are stored explicitly; one is implicit from other parts of the encoding.) So, the conversion must round the exact value of .99 to something that fits in 24 bits. This produces 1.fae148p-1. If you write 1.fae147ae147ae147… in binary and count out 24 bits, you can see where the rounding occurs. Here they are with the first 24 bits in bold: 1.11111010111000010100011110101110000101000111…p-1. When rounding, we look at the bits being removed, see that they are more than half the lowest bit being kept (the first bit being removed is 1, and there are additional 1 bits beyond it), and decide to round up. So rounding produces 1.11111010111000010100100p-1. In hexadecimal, that is 1.fae148p-1.

When .99 is converted to double, 53 bits of the significand can be stored. So it is rounded at a different position. This produces 0x1.fae147ae147aep-1.

The two values are different, comparing them directly would report they are different, and converting them from the floating-point format to hexadecimal numerals produces different results.

Eric Postpischil
  • 195,579
  • 13
  • 168
  • 312
  • many thanks Eric, can you please read this question [http://stackoverflow.com/questions/19590878/java-double-in-place-of-float] and see my edits, please let me know if you agree with the finding. Many thanks again. – jakstack Oct 26 '13 at 18:32
-1

since we are representing a small number like 0.99 we should get the same hex value regardless. Agree?

It's a small precision in decimal, but, as you can see, its hex representation repeats (0x1.f(ae147)*). It is being represented exactly, but it cannot be printed exactly.

Kevin
  • 53,822
  • 15
  • 101
  • 132
  • thanks, can you please clarify "It's a small precision in decimal." and why can't it be printed exactly? Also can't see how it repeats, I can see that up to 0x1.fae14 is the same but the rest is different. – jakstack Oct 26 '13 at 15:21
  • "it's a small precision in decimal" means that the representation in base-10 only needs two digits of precision, that can be printed exactly. What I was saying about repeating is that in the hex representation, the `ae147` part repeats. The number is really `0x1.fae147ae147ae147ae147ae147ae147ae147ae147ae147ae147ae147ae147ae147...`. It's just being printed to a few digits. – Kevin Oct 26 '13 at 15:25