0

When I am doing a floor( 10000000 * 1.1158 ) the output is 11157999 instead of being 11158000. But when i try a floor( 11158000 ) it returns me the good value.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/">
  <html>
  <body>
format-number(floor(10000000 * 1.1158) div 10000000, '#.0000000') = <br/>
     <xsl:value-of select="format-number(floor(10000000 * 1.1158) div 10000000, '#.0000000')"/>
<br/><br/>
floor(10000000 * 1.1158) div 10000000 = <br/>
     <xsl:value-of select="floor(10000000 * 1.1158) div 10000000"/>
<br/><br/>
floor( 10000000 * 1.1158 ) =<br/>
     <xsl:value-of select="floor( 10000000 * 1.1158 )"/>
<br/><br/>
10000000 * 1.1158 = <br/>
     <xsl:value-of select="10000000 * 1.1158"/>
<br/><br/>
floor(11158000 =
<br/>
     <xsl:value-of select="floor(11158000)"/>
</body>
  </html>
</xsl:template>
</xsl:stylesheet>

Output:

format-number(floor(10000000 * 1.1158) div 10000000, '#.0000000') =
1.1157999

floor(10000000 * 1.1158) div 10000000 = 
1.1157999

floor( 10000000 * 1.1158 ) =
11157999

10000000 * 1.1158 = 
11158000

floor(11158000 = 
11158000
Mike
  • 391
  • 2
  • 16
  • 3
    See http://stackoverflow.com/questions/12746624/number-rounding-and-precision-problems-in-xslt-1-0 and http://stackoverflow.com/questions/3805248/xsl-rounding-format-number-problem – Gabriele Petrioli Apr 19 '16 at 10:04

1 Answers1

1

If you are using XSLT 1.0 then all arithmetic is double-precision floating point. This can't represent all decimal fractions exactly, so it uses an approximation. For example the nearest xs:double value to 1.1158 is probably something like 1.1157999999999. If you want exact decimal arithmetic, you'll need to use the xs:decimal data type in XSLT 2.0.

Michael Kay
  • 156,231
  • 11
  • 92
  • 164
  • This is why when working with floating point, it pays to add a small value slightly greater than the algorithm error to one of the values, thus ensuring that the result can be simply trimmed. – Patanjali Aug 01 '20 at 07:55