C# precision calculations multiplying int and double

Question

Consider this code:

Console.WriteLine((197688 * 0.1) == 19768.800000000003); // True
Console.WriteLine((197688 * 0.1) == 19768.8); // False

is an incorrect calculation?

Java-> http://ideone.com/D3QsUo C# -> http://ideone.com/wp5pM9

[What Every Computer Scientist Should Know About Floating-Point Arithmetic](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) — Steve, Jan 07 '14 at 18:16
Been discussed a lot of times; floating-point is not perfect, you need to compare with a delta. — Pierre-Luc Pineault, Jan 07 '14 at 18:17

score 3 · Accepted Answer · answered Jan 07 '14 at 18:20

Floating-point types (and calculations) are imprecise in nature. They work in binary, not in decimal, and hence calculations yield “unexpected” decimal values.

Compare it with this:

Console.WriteLine((197688 * 0.1m) == 19768.800000000003m); // False
Console.WriteLine((197688 * 0.1m) == 19768.8m); // True

The result is as you'd expect, because decimal is used. As the name suggests, it's suited for decimal calculations because of its internal representation.

Side note: The rule of thumb is never use floating-point types for monetary calculations. Exactly becuase of precision issues with decical values.

Also another rule of thumb is to never use `==` with floating point types to keep from falling in to the same issue too. — Scott Chamberlain, Jan 07 '14 at 18:20

score 0 · Answer 2 · answered Jan 07 '14 at 18:20

0

A double only has 15-16 digits of precision - the compiler will translate your literal to the closest value that can be represented by a double which is this case is 19768.8

answered Jan 07 '14 at 18:20

D Stanley

149,601
11
178
240

C# precision calculations multiplying int and double

2 Answers2