0

I'm processing some floating point numbers, adding and subtracting them. This way I end up with numbers that are just a little (0.0000000000000001, 0.000000000000005, etc) off:

float_calc = [0.0]
float_calc.append(float_calc[0] + 0.000317)
float_calc.append(float_calc[1] + 0.000318)
float_calc.append(float_calc[2] - 0.000318)
float_calc.append(float_calc[3] + 0.000317)

print(float_calc)
# [0.0, 0.000317, 0.0006349999999999999, 0.00031699999999999995, 0.000634]

Is there anything I can do about this? How can I make sure that 0.000317 + 0.000318 comes out as 0.0006340.000635?

Walter Vos
  • 436
  • 1
  • 3
  • 13
  • You can't. Neither of the numbers 0.000317 nor 0.000318 can be represented exactly in binary. The are approximations. So, just do your computations and use the proper formatting when you print the values. And I presume you really wanted the answer to be 0.000635, not 0.000634. – Tim Roberts Nov 05 '21 at 20:20
  • Unfortunately as far as I know there is no way to do it. see: https://stackoverflow.com/a/455634/2681662 You can round your numbers or if you want compare them, you should give them a margin like `check - 0.0000001 – MSH Nov 05 '21 at 20:23
  • When working with a reasonable number of decimal digits, there is https://docs.python.org/3/library/decimal.html which may help. – Michael Butscher Nov 05 '21 at 20:25
  • The easy answer is to change your units so you can do everything with ints. – Samwise Nov 05 '21 at 20:38
  • Thanks for taking the time to give me some alternative avenues to explore. For now I'm going with Python's decimal library. So far, seems to work quite well. – Walter Vos Nov 06 '21 at 15:57

0 Answers0