0

I had to scale a double type value and return it with one number in the fraction part. It worked fine in most cases but with number 1740000000 as price value, the integral part equals 1 and the fracton part equals 7 but (fraction * 0.1) equals 0.700000000000007 instead of 0.7. It is strange because 0.7 is not a large number. I somehow handled the problem but I just want to know why this happened???

priceValue = priceValue / 1000000000;
var integral = Math.Floor(priceValue);
var fraction = Math.Floor((priceValue - integral) * 10);
priceValue = integral + (fraction * 0.1)
Fateme12
  • 44
  • 4
  • https://stackoverflow.com/questions/753948/why-is-floating-point-arithmetic-in-c-sharp-imprecise ? – Marc Gravell Mar 20 '20 at 12:22
  • and know I found similar questions thank you. this is helpful too https://stackoverflow.com/questions/588004/is-floating-point-math-broken – Fateme12 Mar 20 '20 at 12:56

0 Answers0