PowerShell calculates wrong

Question

By accident I came across this simple math equation which PowerShell gets wrong.

PS 99.804-99.258
0.546000000000006

PS 10.804-10.258
0.546000000000001

This made me curious and I tried some different calculators and most Calculator gets it right and return

0.546

And:

PS 1.804-1.258
0.546

is correct too. But I found other calculators that also returned wrong results:

0.54600000000001
0.545999999999999999
0.5460000000000065

Why does this happen and how can I get the correct result in PowerShell?

Yeah, while the question linked as a duplicate is present, it's a generic answer that doesn't describe how to resolve the problem with PowerShell (use the `[decimal]` data type). The linked question should not be marked as a dupe here. — Bacon Bits, Jul 30 '18 at 12:24

score 5 · Accepted Answer · edited Jun 20 '20 at 09:12

This is because the default type used in your example is System.Double,

99.804.GetType()

>>>>IsPublic IsSerial Name                                    BaseType
    -------- -------- ----                                    --------
    True     True     Double                                  System.ValueType

In your particular case, what you want to use is System.Decimal which is more precise,

[decimal]99.804-[decimal]99.258

>>>>0.546

or alternatively,

99.804d-99.258d

>>>>0.546

Further information

I felt the need to clarify what I meant by more precise. Basically, it is important to understand that double type has a much more larger range than decimal type. But, larger range doesn't mean more accurate.

Actually, System.Decimal was initially created to take care of currency/money problem where decimal point can be a mess. It's preciseness rely in the fact that it uses base-10 math instead of base-2 math for System.Double. Obviously, the latter is much more efficient to calculate but less accurate.

This can be validated here in the MSDN documentation where it states that,

You can use Decimal variables for money values. The advantage is the precision of the values. The Double data type is faster and requires less memory, but it is subject to rounding errors. The Decimal data type retains complete accuracy to 28 decimal places.

Floating-point (Single and Double) numbers have larger ranges than Decimal numbers but can be subject to rounding errors. Floating-point types support fewer significant digits than Decimal but can represent values of greater magnitude.

For more detail as to _why_ this happens you can look at https://stackoverflow.com/questions/588004/is-floating-point-math-broken?noredirect=1&lq=1 — Matt, Jul 30 '18 at 12:34

PowerShell calculates wrong

1 Answers1