Strange rounding behavior

Question

I'm trying to round doubles to a specific precision, however the following functions give me different results:

Version 1:

static double RoundPrecision(double& val)
{
    val = floor(val * 1000 + 0.5) * 0.001;
    return val;
}

Version 2:

static double RoundPrecision(double& val)
{
    val = floor(val * 1000 + 0.5) / 1000;
    return val;
}

Example output when rounding the number 300.9:

Version 1: 300.90000000000003
Version 2: 300.89999999999998

Both versions sometimes give the same result, but for specific inputs the results differ. I have to have consistent behavior when equating numbers to other variables in the program.

EDIT:

I am aware of the problems with floating point precision, which is exactly what I'm trying to avoid here by rounding. I need a consistent way to round to 3 decimal point precision.

The whole point of rounding is to avoid these built in precision errors, it's just that I need a consistent result — Eyal K., Feb 28 '17 at 09:43
Computers are limited in their precision. You either accept the available precision in `double` or `float`, or you seek other libraries that give you sufficient precision, but with worse performance. Realistically, you cannot achieve exact real-numbers on computers, and you won't even need it. `double` gives you 10^-16 relative precision, then try `long double`. If that isn't enough, then seek external libraries that would do that for you. Here's an example of an arbitrary precision library: https://gmplib.org/ — The Quantum Physicist, Feb 28 '17 at 09:47
I need precision up to 3 decimal points, this is why I have these functions. My problem is that both ways of rounding to that precision give different results, and I need to know which one to use — Eyal K., Feb 28 '17 at 09:50
It doesn't matter which one to use. If you read this number up to 3 decimal points, then both are the same. I think the reasons you're confused is that you don't know how to compare floats. Here's how you do it: http://stackoverflow.com/questions/17333/what-is-the-most-effective-way-for-float-and-double-comparison — The Quantum Physicist, Feb 28 '17 at 09:52

score 1 · Accepted Answer · edited May 23 '17 at 12:09

Computers are limited in their precision. You either accept the available precision in double or float, or you seek other libraries that give you sufficient precision, but with worse performance. Realistically, you cannot achieve exact real-numbers on computers, and you won't even need it. double gives you 10^-16 relative precision, then try long double. If that isn't enough, then seek external libraries that would do that for you. Here's an example of an arbitrary precision library.

From the comments I see that you need 3 decimal places precision. If you read this number up to 3 decimal points, then both results are the same. I think the reasons you're confused is that you don't know how to compare floats. Here's how you do it.

Strange rounding behavior

1 Answers1