4

In my python program, an if statement is not being entered. I have simplified the code to the following:

x = -5
while x < 5:
    if (x == 0):
        print 0
    x += .01

This program does not output anything.

However, changing the last line to x += .5 makes the program output 0. What's the problem?

Dirk
  • 30,623
  • 8
  • 82
  • 102
gsgx
  • 12,020
  • 25
  • 98
  • 149
  • 7
    You may want to read [What Every Computer Scientist Should Know About Floating-Point Arithmetic](http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html) – Dirk Sep 28 '11 at 16:15

7 Answers7

9

Floating point number representation might not be accurate enough. You should never test for zero equality but instead use something along

if (abs(x) < 1E-10) ...
Howard
  • 38,639
  • 9
  • 64
  • 83
  • And `abs(x - y) < epsilon` in general, where `x` and `y` are the values you're comparing and `epsilon` is some tiny number which is smaller than the error margin you care about. –  Sep 28 '11 at 16:15
  • This fix is kind of annoying though. For example, if I want it at a point other than zero I'll have to right .04999 < x < .05001 if I wanted it at .05. There really should be a fix for this coded into the programming language... But thanks, it works for now. – gsgx Sep 29 '11 at 02:08
  • Unfortunately a "fix" is impossible, as "equals" can mean quite different things to different people. Do note, that python folks are aware of this, and provide an explanation: http://docs.python.org/tutorial/floatingpoint.html – Andriy Drozdyuk Sep 29 '11 at 20:23
7

Behold the power of the print statement...

Let us insert a print statement...

x = -5
while x < 5:
    if (x == 0):
        print 0
    x += .01
    print x

Running this program, and inspecting the output around 0 reveals the problem:

...
-0.13
-0.12
-0.11
-0.1
-0.0900000000001
-0.0800000000001
-0.0700000000001
-0.0600000000001
-0.0500000000001
-0.0400000000001
-0.0300000000001
-0.0200000000001
-0.0100000000001
-6.23077978101e-14
0.00999999999994
0.0199999999999
0.0299999999999
0.0399999999999
0.0499999999999
0.0599999999999
0.0699999999999
0.0799999999999
0.0899999999999
0.0999999999999
0.11
0.12
0.13
...

Oh boy, it is never actually equal to zero!

Solutions:

  1. Use integers. Most reliable.

    x = -500    # times this by a 100 to make it an integer-based program
    while x < 500:
      if (x == 0):
        print 0
      x += 1
    
  2. Never test equality with floating point arithmetic, but rather use a range:

    delta = 0.00001 #how close do you need to get
    point = 0 #point we are interested in
    if  (point-delta) <= x <= (point+delta):
        # do stuff
    
Andriy Drozdyuk
  • 58,435
  • 50
  • 171
  • 272
  • `epsilon` is a better name choice for this tiny margin than `delta`, as it by convention represents an infinitesimal differential amount, often corresponding to some desired degree of precision. `delta` is usually used as the difference between two values (observed-expected, or currentvalue-previousvalue). Not as strong a convention as using i, j, and k for loop counters, but still a nice distinction to keep. – PaulMcG Sep 28 '11 at 22:40
  • @PaulMcGuire: By convention in mathematics, both epsilon and delta are used for tiny quantities. I would argue that delta is a better choice actually, because "epsilon" has a specific meaning in the domain of floating-point arithmetic, which is distinct from how it is used here. An even better name would be `comparisonThreshold`, however =) – Stephen Canon Sep 29 '11 at 12:22
  • @StephenCanon: Are you sure? My engineering classes all used epsilon for this purpose (maybe this is an "engineers say epsilon, scientists say delta" thing). A quick Googling of "floating point epsilon" and "floating point delta" seems to turn up more hits for "epsilon", and the top few all describe its use for just this purpose. Even the "What Every Computer Programmer..." citation refers to epsilon, or specifically "machine epsilon", as the smallest discernable difference in the FPU. What is the "specific meaning in the domain of FP arithmetic" you are thinking of? – PaulMcG Sep 29 '11 at 12:38
  • All of the hits for "floating point epsilon" that you're finding are precisely the problem; "floating-point epsilon" or "machine epsilon" refers to one specific value -- the difference between 1.0 and the next representable value. This is far to small to be generally useful for the sort of comparison that you propose here, and the use of the name "epsilon" will lead to confusion. Better to use a name without that baggage. – Stephen Canon Sep 29 '11 at 12:51
  • @PaulMcGuire: As a side note, surely an engineer remembers "epsilon-delta proofs" from freshman calculus? "delta" is frequently used for a tiny number in mathematics. (Of course, it is *also* used for distances, as you note, which is why I suggested a completely baggage-free english name) – Stephen Canon Sep 29 '11 at 12:53
3

This is a rounding issue - decimal values cannot be represented exactly in binary, so x never exactly equals 0.0000000000....

try replacing if (x == 0): with if -0.001 < x < 0.001:

BTW, the parentheses are unnecessary in a python if statement.

edit: Printing out the values between -1 and 1 in steps of 0.01 shows this is the case - where zero should be it prints 7.52869988574e-16.

Dave Kirby
  • 25,806
  • 5
  • 67
  • 84
  • 3
    _Some_ decimal values can be exactly represented in binary. `0.1` is not one of those values. – agf Sep 28 '11 at 16:14
  • @agf: yes, I should have made that clearer. This is why 0.5 works but 0.1 does not - 0.5 is exactly representable in binary. – Dave Kirby Sep 28 '11 at 16:22
3

Others have pointed out the issue with floating-point numbers being unable to represent values exactly. If you need exact decimal representation of a number, you can use the Decimal class:

from decimal import Decimal

x = Decimal(-5)
while x < 5:
    if (x == 0):
        print 0
    x += Decimal(".01")

This will print 0 as you expect.

Note the use of a string for the increment. If you used Decimal(.01) you'd have the same problem with accurate representation of 0.01, because you're converting from a floating-point number and have already lost the accuracy, so the class doesn't allow that.

kindall
  • 178,883
  • 35
  • 278
  • 309
  • This is the right thing to do. OP should never do any scientific computations on `float` type. – patrys Sep 28 '11 at 18:22
0

In binary, .01 has no exact representation but .5 does.

It's the same problem you'd have in decimal if you represented 1/3 as .333333 and kept adding 1/3 until you reached 1. After three additions, you'd have .999999 which is not exactly equal to 1.

Don't compare non-integers for equality unless you precisely understand the rules for doing so and are 100% sure your case is one of the ones that will work.

David Schwartz
  • 179,497
  • 17
  • 214
  • 278
0
-0.01
7.52869988574e-16
0.01

I suggest you say x > -.001 and x < .001 or something of the sort

pyInTheSky
  • 1,459
  • 1
  • 9
  • 24
0

To be precise to your query, floating point numbers are stored in computer hardware in binary(base 2) fractions. So even if you store some floating point like 0.01 in a variable, the computer would ultimately convert that into it's equivalent binary value. For your convenience, conversion of 0.01 float to binary:

0.01 * 2 = 0.02 [0]
0.02 * 2 = 0.04 [0] 
0.04 * 2 = 0.08 [0]
0.08 * 2 = 0.16 [0]
0.16 * 2 = 0.32 [0]
0.32 * 2 = 0.64 [0]
0.64 * 2 = 1.28 [0]
0.28 * 2 = 0.56 [0]
0.56 * 2 = 1.12 [1]
...

This calculation would be too lengthy to show here in full and probably won't end at all. But the fact i want to state here is that, most fractional decimals cannot be exactly converted into binary fractions. As a result, the decimal point you store will be approximated by the binary floating fractions stored in the machine(which obviously can't store very very long binary value). So when the calculation is done with that value, you certainly shouldn't expect the accurate floating value. That's the case with putting x += 0.01 in your code. However conversion of 0.5 to it's binary equivalent would give:

0.5 * 2 = 1.0 [1]

So binary equivalent of 0.5 float is 0.1. Since it is perfectly represented in binary in your machine. You would get the exact result.

It has nothing to do with your code or python. It's just the way computer hardware works :)

consumer
  • 709
  • 3
  • 7
  • 19