0

Working on a problem that uses a ratio of two parameters. I noticed some weird behavior with the division operator / with values < 1e-1 that have a multiple of three. For example

0.075 / 0.025 0.15 / 0.05 gives the output of 2.9999999999999996. I can't use this resultant in my solution as I have a check to ensure the ratio is a whole number. These parameters should work and confused about why this is the output.

Whereas something like 0.9 / 0.3 gives the correct output.

Hello123
  • 1
  • Floating point numbers are a notoriously difficult thing to handle in a discrete math system. There are a number of posts on why this is, and even more books written about it - but suffice to say you might need to round. – Nathaniel Ford Jan 31 '23 at 17:33
  • See also: [Why does the floating-point value of 4*0.1...](https://stackoverflow.com/questions/39618943/why-does-the-floating-point-value-of-40-1-look-nice-in-python-3-but-30-1-doesn?noredirect=1&lq=1) – Nathaniel Ford Jan 31 '23 at 17:36

0 Answers0