0

Can someone please explain why the calculated value 180 * 2.2 will be different when stored in a decimal vs. a double; and perhaps also explain why .NET is calculating 180 * 2.2 with a decimal point in the first place?

  • Standard Calculator: 396
  • Expression (Double): 396.00000000000006
  • Expression converted to decimal: 396

Visual Studio Watch

contactmatt
  • 18,116
  • 40
  • 128
  • 186

1 Answers1

1

Doubles are not an exact representation: there is a lot of rounding and truncation (but you do get a much greater range).

Reading: What Every Computer Scientist Should Know About Floating-Point Arithmetic.

Richard
  • 106,783
  • 21
  • 203
  • 265