1

How can I round var x = 0.10000000000000001 to 0.10?

phuclv
  • 37,963
  • 15
  • 156
  • 475
pether
  • 11
  • 3

3 Answers3

0

You really don't need that.
Both values are essentially the same value 0.100000000000000005551115123126 due to the limited precision of floats.

You can see binary representation here and here

Perhaps you want to show truncated value - in this case use formatting functions of your programming language, like this in C#:

 Console.WriteLine(x.ToString("0.00"));
MBo
  • 77,366
  • 5
  • 53
  • 86
  • Thank you for your answer. I do ask to have a better visual representation but not as a `string`. What I do is to serialize a `double` variable to `json`. `"scale-factor" : 0.29999999999999999` feels a little bit odd. – pether Sep 30 '21 at 20:35
  • OK, I see. I am not familiar with json, but look [at this theme](https://stackoverflow.com/questions/12283070/serializing-a-decimal-to-json-how-to-round-off) about c#/json formatting – MBo Oct 01 '21 at 04:31
  • I think the problem relates more to the programming language instead of json. I would expect a function similar to `Math.round (0.1234, 2)` which would result into `0.1` or something. I know that there is a [math library](https://valadoc.org/glib-2.0/GLib.Math.html) but I haven't found what I need. – pether Oct 01 '21 at 08:05
  • 2
    But... there is no double number that is exactly equal to 0.1 – MBo Oct 01 '21 at 08:33
0

In case you are outputting your data to a FileStream, you can use its method printf(string, ...) to format your data accordingly. This method is actually simply C's well-known fprintf().

So, assuming that you write your JSON data to a FileStream, you can use the following code to truncate (technically not equivalent to rounding, but coming very close) your double values:

FileStream output; // the FileStream you're actually outputting to
double x = 0.1000000000001;

output.printf ("{\"some_value\": %.2f}", x);
// result: {"some_value": 0.10}

This will preserve two decimal places, regardless of their value (meaning that two zeroes will also be output).

In case you need more information on how format strings work, printf()'s documentation is very precise and contains a plethora of examples.

Also, you can see that rounding is actually performed by testing this with your example from one of your comments:

// test code
double x = 2.999999999999d;
stdout.printf ("%.2f", x); // will print 3.00

Try it online!

Alexander Leithner
  • 3,169
  • 22
  • 33
0

The round() function in C rounds to the integer, so the common way to round to a fixed number of decimals is to multiply up then divide the result, e.g. round(x * 100)/100 for two decimal places. There's a good answer on Stack Overflow: How do I restrict a float value to only two places after the decimal point in C?

The important thing to understand is the IEEE 734 floating point arithmetic is not decimal arithmetic. The value 0.1 can't be represented precisely, as the other answers have already stated. See the difference in the example below of rounding 0.123 and 0.10000000000000001.

Compile the following Vala code with valac -X -lm example.vala, the -X -lm tells the compiler to link with the C maths library:

void main () {
    var x = 0.10000000000000001;

    // printf style for serialization to fixed decimal places, outputs 0.10
    // this is a good way to serialize to two fixed decimal places
    print ("%.2f\n", x);

    // common way of rounding in C, outputs 0.12 in this example,
    // but read example afterwards where the result is not as you expect
    var y = 0.1234;
    double a = Math.round(y*100)/100;
    print (@"$a\n");

    // IEEE 754 floating point arithmetic isn't decimal arithmetic,
    // this outputs 0.10000000000000001, which is the same as the input
    double z = Math.round(x*100)/100;
    print (@"$z\n");
}
AlThomas
  • 4,169
  • 12
  • 22