0

I have a Swift extension on NSDecimalNumber:

extension NSDecimalNumber {

    var asRawValue: NSDecimalNumber {
        return self.multiplying(byPowerOf10: 30)
    }

}

And a test:

let t = NSDecimalNumber(value: 335).asRawValue
let u = NSDecimalNumber(value: 0.00001).asRawValue

// fails
XCTAssert(t.subtracting(u).compare((NSDecimalNumber(value: 334.99999)).asRawValue) == .orderedSame)

// console 
po NSDecimalNumber(value: 334.99999).asRawValue
334999990000000051200000000000000

Why is there a random 512 in the middle of the long tail of zeroes of my NSDecimalNumber? How can I avoid this so I can rely on the math of very small numbers?

Thanks

Zack Shapiro
  • 6,648
  • 17
  • 83
  • 151
  • The `334.99999` is a `Double`. And that value isn't exact as a `Double`. – rmaddy Jan 18 '18 at 20:25
  • Oh, was this my bad? – Zack Shapiro Jan 18 '18 at 20:28
  • You could create the `NSDecimalNumber` from a `String`: `po NSDecimalNumber(string: "334.99999").asRawValue` – rmaddy Jan 18 '18 at 20:30
  • 1
    This is related to point 2 in my answer to [your other question](https://stackoverflow.com/q/48308639/1271826): The 334.99999 `Double` cannot be represented perfectly in decimal, and thus suffers the 16 significant digit limitation of `Double`. If, however, you use `NSDecimalNumber(mantissa: 33499999, exponent: -5, isNegative: false)`, that can be. Bottom line, stop passing floating point values to `NSDecimalNumber` (or `Decimal`) and you won't see these sorts of artifacts. – Rob Jan 18 '18 at 20:49

0 Answers0