java floating point error when reading from binary file

Question

I have a floating point number stored in a binary file 0.9999833107. When I read the number in it is either truncated or extended depending on whether I use float or double to store it. I read in the binary with the following code:

public double readFloat(RandomAccessFile fp){

    byte[] data = new byte[4];

    try {

        data[0] = fp.readByte();
        data[1] = fp.readByte();
        data[2] = fp.readByte();
        data[3] = fp.readByte();

    } catch (IOException e) {
            e.printStackTrace();
            return 0;
    }

    ByteBuffer buffer = ByteBuffer.wrap(data);

    return buffer.getDouble();
}

This method returns 0.9999833106994629. When I change the method to output a float the value returned is 0.9999833. Does anyone have any idea how I can get the number out that was written into the file?

You'll want to check [this](http://stackoverflow.com/questions/322749/retain-precision-with-doubles-in-java) out. — Fritz, Oct 11 '12 at 21:23
How have you got the information that it's *exactly* 0.9999833107? It sounds like that's an approximation to me... although given that there are only 4 bytes, I'd *expect* it to be `float`... and a `float` can't represent that many digits accurately... — Jon Skeet, Oct 11 '12 at 21:26
I decided to read the bytes and then output them so I could see what was actually stored in the file. Turns out {63, 127, -2, -24} was stored which resolves to 0.9999833 so when I read as a float and return a float I get the real value. I can also reproduce the additional 107 by forcing the output to be 10 digits with (%.10f). So the additional characters are phantoms created by the system trying to store an approximate value in a double. — Peter, Oct 12 '12 at 00:13

score 3 · Accepted Answer · answered Oct 11 '12 at 21:44

If you're only storing 4 bytes, you could only possibly have 9 guaranteed significant digits of accuracy, as there are only 4294967296 values representable within 32 bits - and that's assuming you could use all the bits as mantissa, with nothing for exponent or sign.

I think it's far more likely that this was originally a value "close to" 0.9999833107, represented as a double, and then truncated to a float for storage. Note that 0.9999833107 itself can't be stored exactly as a double - the closest double value to that is 0.9999833107000000165243136507342569530010223388671875.

In other words, when reading it as a float you are getting "the number that was written into the file". It may not have been the number you had before the conversion for storage, but it's the number that was actually written into the file.

java floating point error when reading from binary file

1 Answers1