Arithmetic Error

Question

Possible Duplicate:
Why does this subtraction not equal zero?

I just came across a weird situation. Why is below code not returning 0? Instead it's returning a very small negative number.

<cfset x = 5448.10-3311.23-2136.87>
<cfoutput>x=#x#</cfoutput>

Above code will output: x=4.54747350886E-013

I ran above code both in CF9 and CF10 with the same results.

Any help is greatly appreciated.

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems — Edvard Pedersen, May 03 '12 at 18:25
@RiverC - +1. As you suspected, yes. If you are not familiar with CF it is mostly typeless, but is built atop java. When math operators are used, numeric values are implicitly converted to `java.lang.Double`. — Leigh, May 04 '12 at 02:52

score 2 · Accepted Answer · answered May 03 '12 at 21:53

2

<cfset x = PrecisionEvaluate(5448.10-3311.23-2136.87)>
<cfoutput>x=#x#</cfoutput>

answered May 03 '12 at 21:53

Henry

score 0 · Answer 2 · answered May 03 '12 at 18:36

As others have stated it is Floating point accuracy related. I just wanted to point out a resolution if you are trying to get a reasonable result

<cfset x1 = 5448.19-3311.23-2136.87 />
<cfset x2 = numberformat(x1, "9.99") />
<cfoutput>x1=#x1#<br />x2=#x2#</cfoutput>

Result

x1=0.0899999999997
x2=0.09

The numberformat function will round the number to the specified decimal place when given mask.

2 Answers2