How Floating Point precision setting works

Question

I have seen many answers on how to set precision for floating point numbers and everywhere we are doing something like below :

double RoundDouble(double doValue,int nPrecision)
{
    return (floor((doValue*pow(10,nPrecision)+0.5))/pow(10,nPrecision));
}

I could not understand how multiplying and dividing by almost equal numbers will set precision correctly ? can anybody explain in detail

you can look for explanation at http://stackoverflow.com/questions/588004/is-floating-point-math-broken — Andrew Kashpur, Mar 07 '17 at 11:02
Limits of f.p. precision was not build by "bad ugly man", by has deep reason. To be the true, hard to speak about precision in 'decimal' terms, not that way — Jacek Cz, Mar 07 '17 at 11:08
By "precision", I believe the author refers to decimal places, and _not_ significant figures. — paddy, Mar 07 '17 at 11:18

ilotXXI · Answer 1 · 2017-03-07T12:30:59.667

0

Let's do it step-by-step.

x = doValue * pow(10, nPrecision) — nPrecision digits are shifted to integral part, others stay in fractal part;
y = floor(x + 0.5) — round to integral part (if x is non-negative);
z = y / pow(10, nPrecision) — shift nPrecision digits back to fractal part.

edited Mar 07 '17 at 12:30

answered Mar 07 '17 at 11:10

ilotXXI

1,075
7
9

1

The standard for rounding negatives is to share equal weight across the zero region, and so -0.49 should round to 0. Read more about [std::round](http://en.cppreference.com/w/cpp/numeric/math/round). So `floor(x + 0.5)` is actually correct. – paddy Mar 07 '17 at 11:16
@paddy Yes, I've mixed with `int(x + 0.5)`. Thank you. – ilotXXI Mar 07 '17 at 12:30

score 0 · Answer 2 · answered Mar 07 '17 at 11:24

0

I have not implemented above but If we debug it using the some sample input then something like below will happen:

// say we have 5.89162 and we want it to 2 decimal places 5.89 so
RoundDouble(5.89162,2)
{
return (floor(5.89162*pow(10,2)+0.5))/pow(10,2);
/* which will look like
    floor((5.89162x100+0.5)/100)
     floor(589.662/100)
     floor(5.89662)
and floor function will bound it to 5 it means the output will be 5 instead of 5.89*/
}

answered Mar 07 '17 at 11:24

Adnan Mohib

331
5
16

you have not checked brackets correctly pow(10,2) is not part of floor,so this will return 5.89 not 5 – PapaDiHatti Mar 08 '17 at 03:49

score 0 · Accepted Answer · answered Mar 07 '17 at 11:25

This is just using whole-number rounding to achieve the trick that we all learned at school for cutting off at some number of decimal places: take the first N digits after the decimal, and round the right-most one up or down.

If you think about this in rounding 12.3456 to 2 decimal places, you naturally expect the result to be 12.35, because the '4' is the right-most of the two digits and gets rounded up by the '5' that follows.

Now, to achieve this with math we utilize floor to achieve the rounding (actually you can utilize std::round instead). But that will take us to a whole number and we'll lose all the fractional part.

To avoid that, we first multiply by 100, moving all the interesting parts into the whole number realm:

1234.56

If you round this number to the nearest whole value with either std::floor(x+0.5) or std::round(x), you then get:

1235.0

Finally, dividing this by 100 gives you the number rounded (yes, remember we rounded it) to two decimal places:

12.35

Hopefully you now see what's going on with the call to pow. By raising 10 to the power of nPrecision, we get a scaling factor that will provide that many decimal places after rounding, when using this trick. In this case, we wanted 2, and pow(10,2) is 100.

I took the liberty of cleaning up your function for readability:

double RoundDouble(double doValue, int nPrecision)
{
    double scale_factor = pow(10.0, static_cast<double>(nPrecision));
    return std::round(doValue * scale_factor) / scale_factor;
}

but how it is ensured that double that is returned will exactly contain 12.35 and not 12.350000001 — PapaDiHatti, Mar 07 '17 at 12:57
Yes that is something i want to know because i am trying to convert 50.28999999999999 upto 9 decimal places and still getting 50.289999999 in calling function — PapaDiHatti, Mar 07 '17 at 13:15
In that case, you could use `std::ostringstream` with IO manipulators: `std::ostringstream ss; ss << std::fixed << std::setprecision(nPrecision) << doValue; std::string strval( ss.str() );` — paddy, Mar 08 '17 at 23:47

score 0 · Answer 4 · edited May 23 '17 at 11:53

0

It doesn't. Floating point doesn't have decimal places. It has binary places, and they are incommensurable with decimal places. All it does is provide an approximation.

For proof see here.

If you want decimal places accurately you have to use a decimal radix.

edited May 23 '17 at 11:53

Community

1
1

answered Mar 07 '17 at 16:48

user207421

305,947
44
307
483

How Floating Point precision setting works

4 Answers4