Why does the floating-point number calculation result in C# have an extra tail?

Asked Jul 07 '21 at 03:54

Active Jul 07 '21 at 03:54

Viewed 44 times

My code snippet as below.

int nStdWorkDays;
double dbStdWorkDaysMin;

nStdWorkDays = 21;
dbStdWorkDaysMin = nStdWorkDays * 0.9;

Here, I found the value of dbStdWorkDaysMin is 18.900000000000002 (other than 18.9) when I debugged and added a watch in Visual Studio.

The error resulted in that '18.9 < dbStdWorkDaysMin' is true!

I wonder why this is happening. What are the similar traps? How can we get the correct calculation result?

Thank you all in advance.

asked Jul 07 '21 at 03:54

Johnson

1

Because double are internally stored as base 2. they cant magically hold unlimited precision – TheGeneral Jul 07 '21 at 03:58
See also https://stackoverflow.com/questions/588004/is-floating-point-math-broken – Jeremy Lakeman Jul 07 '21 at 04:11
1

https://floating-point-gui.de/ - as TL;DR: do not expect binary floating point numbers to be _precise_. if you need precision, use decimal. – Franz Gleichmann Jul 07 '21 at 05:47

0 Answers0