Are C doubles different to .NET doubles?

Question

Comparing some C-code and the F# I'm trying to replace it with, I observed that there were some differences in the final result.

Working back up the code, I discovered that even at there were differences - albeit tiny ones.

The code starts by reading in data from a file. and the very first number comes out differently. For instance, in F# (easier to script):

let a = 71.9497985840
printfn "%.20f" a

I get the expected (to me) output 71.94979858400000000000.

But in C:

a =  71.9497985840;
fprintf (stderr, "%.20f\n", a);

prints out 71.94979858400000700000.

Where does that 7 come from?

The difference is only tiny, but it bothers me because I don't know why. (It also bothers me because it makes it more difficult to track down where my two versions of code are diverging)

Are you sure both values are double? Are you sure the error is not in printing? — Euphoric, Jun 05 '12 at 12:11
floating point arithmetic in .net doesn't have a well defined exact result. It's only an approximation. The lower digits can change at the whim of the compiler and JITer, and just observing them with something like `printfn` can affect them. — CodesInChaos, Jun 05 '12 at 12:13
Use a debugger or something to see the exact bit pattern of the variables. If both the F# and C implementations you have use the same floating-point format and the variables have the exact same bit pattern, then the difference is either in the printing or how the floating-point literal is parsed by the compiler. In either case, since floating-points are only approximations anyway, it might not be anything to worry about, since the difference is quite tiny. — In silico, Jun 05 '12 at 12:14
I see at least three different possibilities: 1) The ToString conversion differs 2) The String to Double conversion in the compiler differs 3) The .net code used a higher precision number (generally common, but very unlikely in this specific case) — CodesInChaos, Jun 05 '12 at 12:15
The value you are seeing in F# is not the exact value. Try the DoubleConverter class here to see the exact value: http://stackoverflow.com/a/3495263/93652 — Botz3000, Jun 05 '12 at 12:32
@Insilico, following your suggestion: found useful info [here](http://stackoverflow.com/q/397692/11410). — Benjol, Jun 06 '12 at 11:19

score 7 · Accepted Answer · answered Jun 05 '12 at 12:16

7

It's a diifference in printing. Converting that value to an IEEE754 double yields

Prelude Text.FShow.RealFloat> FD 71.9497985840
71.94979858400000694018672220408916473388671875

but the representation 71.949798584 is sufficient to distinguish the number from its neighbours. C, when asked to print with a precision of 20 digits after the decimal point converts the value correctly rounded to the desired number of digits, apparently F# uses the shortest uniquely determining representation and pads it with the desired number of 0s, just like Haskell does.

answered Jun 05 '12 at 12:16

Daniel Fischer

181,706
17
308
431

So if I perform multiple operations on lists of these numbers, are these differences going to start become significant? I'm trying to work out how I can test to see if this is the case or not... – Benjol Jun 05 '12 at 12:46
That depends. I think .NET gives the JITter quite a bit of leeway, so the results wouldn't necessarily be reproducible across different runs of the .NET code. If the C implementation doesn't claim to follow IEC 60559, it has a lot of leeway too. The question is whether C and .NET produce the same results for the same functions with the same inputs. If they do, you will only ever observe the difference of the string representation, which only appears behind the portion necessary to determine the `double` value, nothing can accumulate. If they don't differences of 1 or 2 ULP in the results ... – Daniel Fischer Jun 05 '12 at 12:57
can accumulate. But unless you do a **lot** of operations (or some badly conditioned operations), the differences will remain small. – Daniel Fischer Jun 05 '12 at 12:59
thanks, I'm now running them side by side, slowly noting at every step how far they diverge... We will see. – Benjol Jun 05 '12 at 13:17
2

@Benjol: To clarify what Daniel Fischer said: as long as both .NET and the C runtime follow the IEEE-754 standard (also known as ISO/IEC 60559), then the actual values computed will be the same. The difference you're observing here is that the runtimes *print* the value differently, not that there is an actual difference in the value represented. So no amount of additional operations can cause this error to accumulate, since the "error" (or, more strictly speaking, the discrepancy) only appears when the result is converted to decimal for display purposes. – Daniel Pryden Jun 05 '12 at 17:23

score 4 · Answer 2 · answered Jun 05 '12 at 21:16

It is the .NET System.Double.ToString() method that is the difference, the method that converts a double to a string. You can have a look at the relevant code by downloading the CLR source as provided in SSCLI20. The conversion is done by clr/src/vm/comnumber.cpp, COMNumber::FormatDouble() function. Which looks like this, the comment in the code is the most descriptive of what's going on:

//In order to give numbers that are both friendly to display and round-trippable,
//we parse the number using 15 digits and then determine if it round trips to the same
//value.  If it does, we convert that NUMBER to a string, otherwise we reparse using 17 digits
//and display that.

The C runtime library doesn't have that feature.

Thanks Hans, that's a fairly conclusive answer. I definitely am getting cumulative errors as I proceed through my algorithms, now I've just got to try and track down exactly where and why... Currently can't work out if it's this floating-point stuff, or plain errors in my code. — Benjol, Jun 06 '12 at 10:33

score 3 · Answer 3 · answered Jun 05 '12 at 12:17

It's just different rounding. The numbers are the same (according to CPython, at least):

>>> '%.44f' % 71.94979858400000000000
'71.94979858400000694018672220408916473388671875'
>>> '%.44f' % 71.94979858400000700000
'71.94979858400000694018672220408916473388671875'

score 0 · Answer 4 · answered Jun 05 '12 at 12:29

Other answers adequately explain the source of the problem (double precision and rounding).

If your numbers are generally of moderate magnitude and decimal precision is very important (moreso than speed of calculation) then perhaps consider using the .NET decimal format. This gives you 28-29 precise decimal places of accuracy without fractional binary rounding errors like double does. The restriction is that the range is smaller (no large exponents!).

http://msdn.microsoft.com/en-us/library/364x0z75%28v=vs.100%29.aspx

score 0 · Answer 5 · edited May 23 '17 at 12:12

Further info for anyone stumbling on this.

Using bits of code found here, I have confirmed (I believe) the assertion that the underlying binary representation (at least for this particular number) is the same.

Here are code samples - note the 'multiplying zero by zero' to eliminate negative zero - which is ugly when converted to long.

//(C# this time)
var d = 71.9497985840;   //or other incoming double value
if(d == 0) d = d * d;    //for negative zero
var longval = System.BitConverter.DoubleToInt64Bits(d); // = 4634763433907061836

In C:

double d;
long long a;
d = 71.9497985840;      //or other incoming double value
if(d == 0) d = d * d;   //for negative zero
a = *(long long*)&d;    //= 4634763433907061836

update - I followed through, and discovered that the discrepancy was being introduced during matrix inversion, because each system called out to a different library, implementing inversion in a different way...

Are C doubles different to .NET doubles?

5 Answers5