39

I've been fighting decimal precision in C# coming from a SQL Decimal (38,30) and I've finally made it all the way to a rounding oddity. I know I'm probably overlooking the obvious here, but I need a little insight.

The problem I'm having is that C# doesn't produce what I would consider to be consistent output.

decimal a = 0.387518769125m;
decimal b = 0.3875187691250002636113061835m;

Console.WriteLine(Math.Round(a, 11));
Console.WriteLine(Math.Round(b, 11));
Console.WriteLine(Math.Round(a, 11) == Math.Round(b, 11));

Yields

0.38751876912
0.38751876913
False

Uhh, 0.38751876913? Really? What am I missing here?

From MSDN:

If the digit in the decimals position is odd, it is changed to an even digit. Otherwise, it is left unchanged.

Why am I seeing inconsistent results? The additional precision isn't changing the 'digit in the decimals position'...

Bennett Dill
  • 2,875
  • 4
  • 41
  • 39
  • 4
    It's rounding a value <= 0.5 down, and > 0.5 up. (i.e. "0.50000" is rounded down, and "0.50002..." is rounded up) - isn't this what you expect? See http://msdn.microsoft.com/en-us/library/ms131274.aspx – Jason Williams Apr 12 '12 at 16:13
  • @Jason Williams: Not quite. It rounds down if < 0.5, up if > 0.5, and then uses banker rounding if == 0.5. – jason Apr 12 '12 at 16:21
  • 18
    You are expecting the round function to round to *the number that is farther away*? Can you explain why would you expect that under *any* circumstances? (This is not a rhetorical question; I am interested to learn why people believe very strange things. This question is actually quite common and yet I still do not understand why so many people believe that *round* should round to *the number that is farther away*.) – Eric Lippert Apr 12 '12 at 16:35
  • 11
    @EricLippert I believe the common misconception stems from the fact that primary-school math classes teach rounding by looking *at the next digit*. We learn to round up if that digit is 5. Then we learn about banker's rounding, and many assume incorrectly that "round to the nearest even digit" applies whenever the next digit is 5. Of course, it applies only when the next digit is 5 *and there are no more nonzero digits after that.* Even the documentation that Oded quoted is misleading: if you read it strictly, the single non-zero `5` digit in `1.0005` would result in a return value of `2`. – phoog Apr 12 '12 at 16:58
  • 1
    I guess my issue has to do with misinterpreting the MSDN documentation. It appears to state "if the digit in the decimals position". For me this implies, "we're only looking at that digit to make our decision". What you're saying makes sense, but I just don't see that in the documentation. Even after looking again. I'll reread that document to see if its clearer with your feedback. – Bennett Dill Apr 12 '12 at 16:58
  • @EricLippert the language in MSDN may have been a reaction to the "community content" I added at http://msdn.microsoft.com/en-us/library/zy06z30k.aspx; the old language was more incorrect: "If the value of the first digit in d to the right of the decimals decimal position is 5, the digit in the decimals position is rounded up if it is odd, or left unchanged if it is even." – phoog Apr 12 '12 at 16:59
  • 11
    But the "round to even" rule is a *tiebreaking* rule; you don't apply a tiebreaking rule *when there already is a clear winner*. – Eric Lippert Apr 12 '12 at 17:49
  • @EricLippert I understand how it works; I am just explaining the thought process that seems to prevail among the people I've discussed this with. The language on MSDN doesn't help, though the new language is considerably better than the old. Fortunately, the method behaves correctly. – phoog Apr 12 '12 at 18:50
  • @phoog: I'm not sure how you get that. Let's say you invoke `Math.Round(1.0005m, 3)`. Using the same notation as the documentation, `d` is `1.0005m` and `decimals` is `3`. The `decimals` position is the third zero (i.e., in `1.0_1 0_2 0_3 5`, it is the `0_3`). There is a single non-zero digit to the right of `0_3`, and its value is `5`. The digit in the `decimals` position (i.e., `0_3` again) is even, and therefore it is left unchanged. Thus, `Math.Round(1.0005m, 3)` evaluates to `1.000m`. I think that you confused "`decimals` decimal position" with "the decimal point." Is that what happened? – jason Apr 13 '12 at 02:49
  • @Jason but suppose you call `Math.Round(1.0005m, 0);`. The `decimals` position is the units digit, `1`. There is a single non-zero digit to the right of this position, and its value is 5. The digit in the `decimals` position is odd, so it is increased from `1` to `2`. Obviously, this interpretation depends on reading "single non-zero digit" as "any number of zeros plus one non-zero digit" rather than "a single digit that is not zero". Because this is a reasonable interpretation of the phrase, the language is misleading. – phoog Apr 13 '12 at 02:53
  • @phoog: I understand what you mean now. However, it's not "single non-zero digit" -> "a single digit that is not zero." Rather, it's "single non-zero digit" -> "a single digit and that digit is not zero." But really it, the documentation should specify that we are talking about some sort of canonical representation (i.e., so we exclude the possibility of a representation like `1.00050`). – jason Apr 13 '12 at 03:02
  • @Jason precisely. If we're going to discount the infinite sequence of trailing zeros, we might as well say "single digit" rather than "single non-zero digit". It's also worth noting that the `decimal` data type distinguishes between `1.5` and `1.500`, which makes "single digit that is not zero" all the more confusing. – phoog Apr 13 '12 at 03:08
  • see also http://stackoverflow.com/questions/311696/why-does-net-use-bankers-rounding-as-default – Ian Ringrose Apr 13 '12 at 08:35

4 Answers4

47

From MSDN:

If there is a single non-zero digit in d to the right of the decimals decimal position and its value is 5, the digit in the decimals position is rounded up if it is odd, or left unchanged if it is even. If d has fewer fractional digits than decimals, dis returned unchanged.

In your first case

decimal a = 0.387518769125m;
Console.WriteLine(Math.Round(a, 11));

there is a single digit to the right of the 11th place, and that number is 5. Therefore, since position 11 is even, it is left unchanged. Thus, you get

0.38751876912

In your second case

decimal b = 0.3875187691250002636113061835m;
Console.WriteLine(Math.Round(b, 11));

there is not a single digit to the right of the 11th place. Therefore, this is straight up grade-school rounding; you round up if the next digit is greater than 4, otherwise you round down. Since the digit to the right of the 11th place is more than 4 (it's a 5), we round up so you see

0.38751876913

Why am I seeing inconsistent results?

You're not. The results are completely consistent with the documentation.

jason
  • 236,483
  • 35
  • 423
  • 525
  • 2
    Just to make sure I'm understanding here, ToEven or AwayFromZero varies as the default based on the precision of the number? – Bennett Dill Apr 12 '12 at 16:48
  • I like your answer and I think it shows what is missing from the MSDN documentation. That is, what happens if it isn't a single non-zero digit. – Bennett Dill Apr 12 '12 at 17:42
  • 5
    @BennettDill "Single non-zero digit equal to 5" is a confusing way of saying "the method's argument is exactly halfway between the two possible return values". If there *isn't* a single digit to the right of the specified place, or if there's only one digit that is not 5, then the argument is not exactly between the possible return values, so you pick the return value that's closer to the argument. Ah, if only we used base 3 numbers, we wouldn't have to worry about this! – phoog Apr 12 '12 at 18:56
  • @ScottChamberlain: Um, I thought Math.Round defaults to ToEven (bankers' rounding). You have to specify you want AwayFromZero (semantic arithmetic rounding) which is not happening here. – KeithS Apr 12 '12 at 19:17
  • That means that if you were to choose AwayFromZero rounding, both of these numbers would round to the same 11-digit value, because the rule you mention is part of the bankers' rounding algorithm. – KeithS Apr 12 '12 at 19:20
26

From MSDN - Math.Round Method (Decimal, Int32):

If there is a single non-zero digit in d to the right of the decimals decimal position and its value is 5, the digit in the decimals position is rounded up if it is odd, or left unchanged if it is even. If d has fewer fractional digits than decimals, d is returned unchanged.

The behavior of this method follows IEEE Standard 754, section 4. This kind of rounding is sometimes called rounding to nearest, or banker's rounding. It minimizes rounding errors that result from consistently rounding a midpoint value in a single direction.

Note the use of single non-zero digit. This corresponds to your first examples, but not the second.

And:

To control the type of rounding used by the Round(Decimal, Int32) method, call the Decimal.Round(Decimal, Int32, MidpointRounding) overload.

Community
  • 1
  • 1
Oded
  • 489,969
  • 99
  • 883
  • 1,009
  • I don't think my issue was with which type of rounding was used. I believe it was not understanding the behavior of the MidpointRounding.ToEven itself. :-( The answers and comments in this question have clearly spelled it out now though, thanks to all! – Bennett Dill Apr 12 '12 at 20:39
4

The part "single non-zero digit in d to the right of the decimals decimal position and its value is 5" explains the result. Only when the part to round is exactly 0,5 the rounding rule comes into play.

Albin Sunnanbo
  • 46,430
  • 8
  • 69
  • 108
4

Let's shift both numbers over 11 digits to the left:

38751876912.5
38751876912.50002636113061835

Using banker's rounding, we round the first one down. Under every midpoint-rounding system, we round the second number up (because it is not at the midpoint).

.Net is doing exactly what we'd expect it to.

BlueRaja - Danny Pflughoeft
  • 84,206
  • 33
  • 197
  • 283