What is assigned to variables in floating point calculations?

Question

I am having some trouble with calculations in Perl (v5.10.1) due to floating point numbers:

#!/usr/bin/perl
use strict;
use warnings;
use POSIX;

my $x1 = 1500;
my $x0 = 1000;
my $dx = 100/3;

print "($x1-$x0)/$dx \n";                                   #(1500-1000)/33.3333333333333
print my $a=(($x1-$x0)/$dx), "\n";                          #15
print my $b=floor(($x1-$x0)/$dx), "\n";                     #14
print my $c=floor($a), "\n";                                #14
print floor(15), "\n";                                      #15
print my $d=floor(sprintf("%.0f", ($x1-$x0)/$dx)), "\n";    #15

Why is the output 14 sometimes? Isn't the value 15 saved as it shows in $a and therefore used floor on the value 15? The comparison of $a and $c leaves me really puzzled...

I read this but can't figure it out. I also found the workaround with sprintf which isn't very handy in my opinion.

score 4 · Answer 1 · answered Nov 04 '13 at 15:57

4

Try:

printf "%.18g\n", my $a=(($x1-$x0)/$dx);

What you see as 15 isn't exactly 15; it may be a little less or a little more. Most floating point numbers can only be represented imprecisely; when used in operations, the effect is, err, multiplied.

Classic reference: http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

The workaround with sprintf is often not so good to use, since it only deals with a very small amount of imprecision. Better to examine your calculations and decide what tolerance you should use and add it before flooring.

answered Nov 04 '13 at 15:57

ysth

96,171
6
121
214

Can you explain the last part more in detail please? What do you mean by "only deals with a very small amount of imprecision"? And can you please give an example of your suggested FP calculation style? The last line of my code uses `sprintf` to adjust a tolerance and uses `floor` afterwards. This is what you suggested?! Just found [this](http://stackoverflow.com/a/1838885/2622293) which leaves me even more confused. – EverythingRightPlace Nov 04 '13 at 20:53

score 2 · Answer 2 · answered Nov 05 '13 at 02:32

In double-precision, the value of $dx is exactly

33.33333333333333570180911920033395290374755859375

The value of ($x1-$x0)/$dx is exactly

14.9999999999999982236431605997495353221893310546875

floor($x1-$x0)/$dx is thus 14.

You get 15 from the print/sprintf because printing rounds the decimal value (unless you ask for more digits, like "%.17g").

What is assigned to variables in floating point calculations?

2 Answers2