5

I'm trying to do some calculations in my python script but i'm getting some weird results. For example:

0.03 // 0.01
>>> 2.0

If I upscale the numbers I get the expected results:

3.0 // 1.0
>>> 3.0

I'm pretty sure that the answer for the first code snippet should be 3.0 and not 2.0. Can someone explain me why this is happening and how to fix it?

flpn
  • 1,868
  • 2
  • 19
  • 31
  • 3
    Floating point rounding error doesn't stop happening just because you use `//`. – user2357112 Jan 08 '20 at 14:21
  • 1
    I'm not sure that this is a duplicate `0.03/0.01 < 3.0` evaluates to `False`. The question seems to involve the semantics of `//` vs. `/` in a way that the proposed duplicate doesn't really explain. – John Coleman Jan 08 '20 at 14:26
  • On the other hand -- the second duplicate target (since added) *is* a genuine duplicate target. – John Coleman Jan 08 '20 at 14:37

2 Answers2

6

This is due to the floating point error. Note that with the above floor division, the remainder is not 0:

0.03 % 0.01
# 0.009999999999999998

So if instead we divide by:

0.03 // 0.009
# 3.0

The answer is correct. Hence 0.03 is not entirely divisible by 0.01 without some remainder due to floating point limitations

yatu
  • 86,083
  • 12
  • 84
  • 139
3

As yatu already mentioned, this is due to floating point errors. Try this instead:

from decimal import Decimal
Decimal('0.03') // Decimal('0.01')
>>> 3
Daan Klijn
  • 1,269
  • 3
  • 11
  • 28