Integer NEQ Int(Integer) Craziness

Question

I'm trying to detect whether I've multiplied the numerator and denominator of a complex fraction enough times (by 10/10) to get integers on both top and bottom. Often my algorithm works. Sometimes, though, the Int() function doesn't work as expected and I get huge numbers out of it. Here is a simplified version of what I'm running:

<cfoutput>
    <cfset RealNumber = RandRange(1000, 10000) / 1000 />
    RealNumber = "#RealNumber#" Int(RealNumber) = "#Int(RealNumber)#"<br />
    <cfloop condition="RealNumber NEQ Int(RealNumber)">
        <cfset RealNumber = RealNumber * 10 />
        RealNumber = "#RealNumber#" Int(RealNumber) = "#Int(RealNumber)#"<br />
    </cfloop>
</cfoutput>

Most of the time this gets what I want, but sometimes it looks like this:

RealNumber = "9.184" Int(RealNumber) = "9"
RealNumber = "91.84" Int(RealNumber) = "91"
RealNumber = "918.4" Int(RealNumber) = "918"
RealNumber = "9184" Int(RealNumber) = "9183"
RealNumber = "91840" Int(RealNumber) = "91839"
RealNumber = "918400" Int(RealNumber) = "918399"
RealNumber = "9184000" Int(RealNumber) = "9183999"
RealNumber = "91840000" Int(RealNumber) = "91839999"
RealNumber = "918400000" Int(RealNumber) = "918399999"
RealNumber = "9184000000" Int(RealNumber) = "9183999999"
RealNumber = "91840000000" Int(RealNumber) = "91839999999"
RealNumber = "918400000000" Int(RealNumber) = "918399999999"
RealNumber = "9.184E+012" Int(RealNumber) = "9.184E+012"
RealNumber = "9.184E+013" Int(RealNumber) = "9.184E+013"
RealNumber = "9.184E+014" Int(RealNumber) = "9.184E+014"
RealNumber = "9.184E+015" Int(RealNumber) = "9.184E+015"

Or even like this:

RealNumber = "2.152" Int(RealNumber) = "2"
RealNumber = "21.52" Int(RealNumber) = "21"
RealNumber = "215.2" Int(RealNumber) = "215"
RealNumber = "2152" Int(RealNumber) = "2152"
RealNumber = "21520" Int(RealNumber) = "21520"
RealNumber = "215200" Int(RealNumber) = "215200"
RealNumber = "2152000" Int(RealNumber) = "2152000"
RealNumber = "21520000" Int(RealNumber) = "21520000"
RealNumber = "215200000" Int(RealNumber) = "215200000"
RealNumber = "2152000000" Int(RealNumber) = "2152000000"
RealNumber = "21520000000" Int(RealNumber) = "21520000000"
RealNumber = "215200000000" Int(RealNumber) = "215200000000"
RealNumber = "2.152E+012" Int(RealNumber) = "2.152E+012"
RealNumber = "2.152E+013" Int(RealNumber) = "2.152E+013"
RealNumber = "2.152E+014" Int(RealNumber) = "2.152E+014"
RealNumber = "2.152E+015" Int(RealNumber) = "2.152E+015"
RealNumber = "2.152E+016" Int(RealNumber) = "2.152E+016"

What am I doing wrong and how do I compensate for this?

score 2 · Answer 1 · edited May 23 '17 at 11:53

2

RealNumber = "9.184E+015"

The output is deceptive. If you use #RealNumber.toString()# the real value is probably something like 9.183999999999998E15. CF uses the approximate type Double for most mathematical operations. So what you are seeing is normal behavior for floating point numbers. If you need greater accuracy, use PrecisionEvaluate. It makes use of BigDecimals for arithmetic operations (because they are more precise than java.lang.Double).

edited May 23 '17 at 11:53

Community

1
1

answered Feb 06 '12 at 22:53

Leigh

28,765
10
55
103

`` still gets the huge numbers. – Dave Babbitt Feb 07 '12 at 13:31
That is because you have to use it on both values ie `precisionEvaluate(RealNumber * 10)`. Otherwise CF falls back to using floating point numbers when you perform math operations. – Leigh Feb 07 '12 at 17:36

score 0 · Answer 2 · answered Feb 06 '12 at 22:05

Try using javacast.. ColdFusion is trying to dynamically determine your variable type and is not doing a great job.. Javacast covers a multitude of sins.

<cfoutput>
<cfset RealNumber = RandRange(1000, 10000) / 1000 />
RealNumber = "#RealNumber#" Int(RealNumber) = "#Int(RealNumber)#"<br />
<cfloop condition="RealNumber NEQ Int(RealNumber)">
    <cfset RealNumber = RealNumber * 10 />
    RealNumber = "#RealNumber#" Int(RealNumber) = "#javaCast("int", RealNumber)#"<br />
</cfloop>

This: `` still gets the huge numbers. – Dave Babbitt Feb 07 '12 at 13:29 — Dave Babbitt, Feb 07 '12 at 13:29

score -1 · Accepted Answer · answered Feb 06 '12 at 22:13

-1

Val() seems to correct it too:

<cfoutput>

    <cfset RealNumber = RandRange(1000, 10000) / 1000 />
    "#val(RealNumber)#" = INT: "#val(Int(RealNumber))#"<br />
    <cfloop condition="val(RealNumber) NEQ Val(Int(RealNumber))">
        <cfset RealNumber = RealNumber * 10 />
        "#val(RealNumber)#" INT: "#val(Int(RealNumber))#"<br />
    </cfloop>

</cfoutput>

answered Feb 06 '12 at 22:13

BKK

2,073
14
17

`` solves the problem and gets the least amount of loop time. – Dave Babbitt Feb 07 '12 at 13:32
@Dave - I am a little skeptical about that ;) PrecisionEvaluate usually results in far fewer iterations because it is not using approximate numbers like 9.183999999. So evaluating 9.184 requires four (4) iterations instead of ~thirteen (13) with `val`. That said, it is not a significant difference overall, unless you are handling hundreds or thousands of numbers. – Leigh Feb 07 '12 at 17:58

Integer NEQ Int(Integer) Craziness

3 Answers3