1

This part of code should speak for itself, my question is can I get 1242.08 result in res variable, and not 1242.07, because in math

b = 1/(1/b)

and in double which has less precision it seems I got good result

Can I fix decimal part of calculation to give me mathematically right result:

decimal rate = 124.2075M;
decimal amount = 10M;
decimal control = decimal.Round(rate * amount, 2); //it is 1242.08

//but
decimal res = decimal.Round(amount * (1 / (1 / rate)), 2);//1242.07 - should be 1242.08
decimal res1 = decimal.Round(amount * 124.2075M, 2);//1242.08 OK

/////////////////////////////////////////////////////////////

//while with double seems OK
double ratef = 124.2075;
double amountf = 10;
double resf = Math.Round(amountf * (1 / (1 / ratef)), 2);//1242.08 OK
double res1f = Math.Round(amountf * 124.2075, 2);//1242.08  OK
Imran Sh
  • 1,623
  • 4
  • 27
  • 50
vldmrrdjcc
  • 2,082
  • 5
  • 22
  • 41
  • You can try Math.Round(1242.075M*100)/100. – Shannon Holsinger Sep 08 '16 at 13:51
  • The identity `b = 1/(1/b)` which is good in pure math (for non-zero `b`), simply does not hold for floating-point arithmetic. In particular, when `rate` is `124.2075M`, then `1M / (1M / rate)` is something else, namely `124.20749999999999999999999991M`. The good thing about `decimal` is that you can clearly see from the printed representation of the value, that it is not the same. (For `double` and `float`, sometimes there is "hidden" precision unless you use a special format string for printing the value.) – Jeppe Stig Nielsen Sep 08 '16 at 14:10
  • Just do this yourself with a piece of paper and use long division to compute 1 / 124.2075 as you learned in elementary school. The paper is not big enough to write the infinite sequence of digits in the result, you have to stop somewhere. And if you do it again with the truncated result you wrote down then you of course don't get the original value back. Computers don't work any differently, they also don't have infinite storage. Your assumption that this should be possible is simply not correct. – Hans Passant Sep 08 '16 at 15:19
  • If you want pretty numbers then multiply everything by a 10000 and use integer math. It is only prettier, it is not more correct. – Hans Passant Sep 08 '16 at 15:22
  • @Hans Passant - it was suprising to me that an erorr on 29th decimal, can lead to an error on the second decimal in the next step of calculation :) – vldmrrdjcc Sep 09 '16 at 06:50

1 Answers1

5

That's a limitation of the datatype decimal that can hold up to 29 digits

The result of the first calculation (res1) does not fit into decimal so you get a invalid/imprecisely result.

decimal rate = 124.2075M;
decimal amount = 10M;

decimal res1 = (1 / rate); //0.0080510436165287925447336111M <- not enough decimal places
decimal res2 = (1 / res1); //124.20749999999999999999999991M
decimal res3 = amount * res2; //1242.0749999999999999999999991M
decimal res4 = decimal.Round(res3, 2); //1242.07M <- correct rounding
fubo
  • 44,811
  • 17
  • 103
  • 137
  • I have noticed that if I round 1/rate on something like 10 digits I got 1242.08, is this by accident? Or can I use something like that to fix this? – vldmrrdjcc Sep 08 '16 at 14:08
  • 1
    That's by accident because you just cut off everything after `0.0080510436`. But that's not a general solution for all numbers - that just works for this one – fubo Sep 08 '16 at 14:14
  • Should I use different data type for this calculation, I saw that with Double I get right result, is it also by accident that double seems to work? – vldmrrdjcc Sep 08 '16 at 14:17
  • 1
    @VladimirRadojicic That is not a complete accident. With `decimal`, the very midpoint where rounding has two equally close "destinations" to choose between, is actually exactly representable. For example `1242.075` (the midpoint between `1242.07` and `1242.08`) is exactly representable as a `decimal`, but not as a `double`. I guess that makes a difference because you round based on decimally "nice" numbers. However, in principle, the same errors occur with binary types like `double`. _Edit:_ A `double` example: `Math.Round(1452.375, 2) != Math.Round(1.0 / (1.0 / 1452.375))` – Jeppe Stig Nielsen Sep 08 '16 at 14:24
  • (I meant `, 2` before the final `)` in above comment, of course.) – Jeppe Stig Nielsen Sep 08 '16 at 14:36
  • Thank you very much on your help, obviously the solution is not to do 1/rate if you don't have to :) – vldmrrdjcc Sep 08 '16 at 15:08