0

for example, I have a code like this

double a = 3;
double b = 1.1;
double c = a * b;

then in the single-step debugging, we can watch that the value of C is not pure 3.3. it will be 3.3000000000000003

is anyone can give me an explanation on this precision's value?

kylejan
  • 163
  • 1
  • 13
  • 2
    `double` is a finite binary fraction. `3/10` can't be precisely represented in binary, for the same reason `1/3` can't be precisely represented in decimal. If you inspect `1.1` closely, it is already inexact. – Igor Tandetnik Dec 26 '14 at 02:28
  • 1
    You might want to cf. http://www.infogoaround.org/JBook/FloatingPoint.pdf (I'm not quoting any of it, because the question has already been resolved as duplicate) – dwn Dec 26 '14 at 02:48
  • Well this is called Rounding Error, which is really important in Numerical computing and Numerical Analysis. – Jake0x32 Dec 26 '14 at 02:49

0 Answers0