.NET decimal rounding - weird result with a Digit repeating lots of times

Question

One of my unit tests failed today. I nailed it down to a rather peculiar decimal rounding issue.

var b = 196.5m;
var result1 = b / 12m;       //16.375
var result2 = b * 1 / 12m;    //16.375
var result3 = b * (1 / 12m);  //16.374999999999999999999999993

What's happening there with result3?

I would very much prefer to use result3 code style (the actual code is much more complex than this obviously) but I want the result3 to be identical to result1 and result2.

-- UPDATE --

I have even rounded it:

Math.Round(196.5m * (1 / 12m), 2, MidpointRounding.AwayFromZero); //16.37
Math.Round(196.5m / 12m, 2, MidpointRounding.AwayFromZero);  //16.38

`var b = 196.5m; b * (1m / 12m);` gives `16.374999999999999999999999993`. What version of .net are you using? — BurnsBA, Jun 03 '21 at 20:35
I get the same as BurnsBA. I am using .Net Core 3.1 on a Mac. Have you considered using `Math.Round(result3, 3);`? — Barns, Jun 03 '21 at 20:39
@BurnsBA Yup, I have double checked, you are correct. I have removed it from the question so it won't confuse anyone. — Rosdi Kasim, Jun 04 '21 at 01:08

BurnsBA · Answer 1 · 2021-06-04T01:45:32.013

This might be relevant, so I want to start by pointing out the Language Specification, section 4.1.7

If one of the operands of a binary operator is of type decimal, then the other operand must be of an integral type or of type decimal. If an integral type operand is present, it is converted to decimal before the operation is performed.

The result of an operation on values of type decimal is that which would result from calculating an exact result (preserving scale, as defined for each operator) and then rounding to fit the representation. Results are rounded to the nearest representable value, and, when a result is equally close to two representable values, to the value that has an even number in the least significant digit position (this is known as “banker’s rounding”). A zero result always has a sign of 0 and a scale of 0.

This tells you results3 and 4 in your test should give identical results. (Also note the "integral" word, implicit float/double conversion is not supported)

Now, in your case you've stumbled upon an equation with some nice simplification properties, in that 196.5 / 12 => 393 * (2/3) / 12 => 131 * 2 / 4. In your results3&4, you calculate the division by 3 first (1/12), which gives 0.0833..., something that can't be exactly represented in decimal. And then you scale 0.0833... up (your order of operations is to divide by 12 then multiple by b).

You can get the same result by first rounding a number that can't be represented in decimal, e.g., something with repeating digits, say 1/7m. For example, 917m * (1 / 7m) = 131.00000000000000000000000004 but note 917m/7m = 131.

You can mitigate this by preferring multiplications first (being careful of overflow). The other option is to round your results.

This is probably a dupe of something like Is C# Decimal Rounding Inconsistent?

or maybe Rounding of decimal in c# seems wrong...

I have actually rounded it, but `Math.Round(196.50m * (1 / 12m), 2, MidpointRounding.AwayFromZero)` returns `16.37` meanwhile the other method returns `16.38`. This is how I stumbled upon this issue, the product price is off by `0.01`. — Rosdi Kasim, Jun 04 '21 at 01:29
@RosdiKasim if you only want to pass unit tests and don't care that `4` will sometimes round up to the next value, you can call Round repeatedly. `var val = 196.5m * (1 / 12m); for (int i=10; i>=2; i--) { val = Math.Round(val, i); } ` => `16.38`. — BurnsBA, Jun 04 '21 at 02:21

score 3 · Answer 2 · edited Jun 04 '21 at 04:31

the problem lies in order of operations and precision of returned values

In your example the first does no multiplication, and the second does the multiplication first and then the division, the 3rd will first do the division, which will return 0,0833333333333333... this will then be represented as 0,0833333333333333 which introduces an imprecision of 3,333...e-17*. this imprecision is then multiplied

When doing stuff like this, try to make sure to first do the multiplications and afterwards the divisions. Doing so will eliminate the risk of multiplying with imprecise fractions

-REMARK-* the exact number here is dependent on which type (decimal, float,...)is returned, which is in turn dependent on which types you are dividing, it likely is not 17

If you want to magnify this issue, try thinking this out in int

.NET decimal rounding - weird result with a Digit repeating lots of times

2 Answers2