4

I tried the following in the REPL:

a = 0.0002 - 0.0001
> 0.0001

b = 0.0003 - 0.0002
> 9.999999999999996e-5

a == b
> false

Why is b not 0.0001 as well? I understand that Float64, like any other type, can have rounding errors, but is this what I'm seeing here?

It's actually a julia package that does this subtraction and check process with some time values (0.0001, 0.0002, ...) that I provide and it crashes because of it. Is there something I can do without modifying the code of that package?

Jan St
  • 41
  • 2
  • 6
    Have a look at `Rational` numbers in Julia. Eg. you can do `3//10_000 - 2//10_000` and then obtain exact results. Additionally there is `approx` eg. try typing in REPL `a ≈ b`. Have also a look at https://stackoverflow.com/questions/34554364/in-julia-is-there-a-way-to-test-for-equality-like-rs-all-equal and https://stackoverflow.com/questions/30264742/exact-decimal-arithmetic-in-julia When you encounter more problems you are welcome to ask a more detailed question! – Przemyslaw Szufel Feb 17 '22 at 01:20

0 Answers0