-2

I have my code as

out = 0.step(10, 0.1 ).to_a
puts out

and when i run it some numbers are output as 5.1000000001, please tell me how to round these numbers so that the output would be 5.1

Output image

SteveTurczyn
  • 36,057
  • 6
  • 41
  • 53
  • [Please do not upload images of code/errors when asking a question.](//meta.stackoverflow.com/q/285551) – Amadan Feb 22 '22 at 15:22
  • Welcome to SO! Please tell us what you've tried beyond the example code. Where did you search? Why didn't that help? Did you write other code? If so, what are the smallest examples that demonstrate your effort to solve this. As is, it appears you haven't tried and want us to point you to documentation, or write code for you. – the Tin Man Feb 22 '22 at 22:47

2 Answers2

0

Use 0r.step(10r, 0.1r) for exact calculation (use .to_f at print time) since Rational numbers don't have the same precision issues that floating point numbers do. E.g.

0r.step(10r, 0.1r).map(&:to_f)

Alternately (and equivalently, for languages that don't have rationals), use integers and divide when you need them:

0.upto(100).map { _1 / 10.0 }

Dividing as the last operation instead of cumulatively adding minimises the floating point error.

While it is much better to not have an error in the first place, using one of the above methods, you could also try to rescue your inexact numbers by rounding:

0.step(10, 0.1).map { _1.round(1) }
Amadan
  • 191,408
  • 23
  • 240
  • 301
0

Alternatively, if the numbers are close enough for the sake of the calculations you're doing, and the issue is only with printing, you can simply format them as you see fit.

out = 0.step(10, 0.1).to_a

out.each do |n|
  printf "%.2f\n", n
end
Chris
  • 26,361
  • 5
  • 21
  • 42