1

Problably an easy questions but this calculation bothers me.. why is double test = 57.05 - 57 equal to 0.049999999999997158 and not 0.05? And which datatype should I use to get the correct result?

  • 1
    but from an *engineering* point of view, this is an error of 0.000000000005 percent, and should be close enough. – pavium Oct 27 '09 at 20:05

4 Answers4

10

Use a decimal. And read this.

MusiGenesis
  • 74,184
  • 40
  • 190
  • 334
4

Try to convert .05 decimal to binary and tell me what result you get (hint: it's a repeating value).

Joel Coehoorn
  • 399,467
  • 113
  • 570
  • 794
2

Long answer: Floating Point Representation

Short answer: types like double conform to IEEE 754, which dictates data types which may have a decimal point anywhere (floating). Because of the representation in the computer's native binary, the result of mathematical operations on these representations in many cases may only be an approximation of the actual result.

Jesse C. Slicer
  • 19,901
  • 3
  • 68
  • 87
1

Floating point calculations are not as correct.

How about double-precision-problems-on-c-net

Community
  • 1
  • 1
Adriaan Stander
  • 162,879
  • 31
  • 289
  • 284