Double precision issues when converting it to a large integer

Question

Precision is the number of digits in a number. Scale is the number of digits to the right of the decimal point in a number. For example, the number 123.45 has a precision of 5 and a scale of 2.

I need to convert a double with a maximum scale of 7(i.e. it may have 7 digits after the decimal point) to a __int128. However, given a number, I don't know in advance, the actual scale the number has.

#include <iostream>
#include "json.hpp"
using json = nlohmann::json;
#include <string>

static std::ostream& operator<<(std::ostream& o, const __int128& x) {
    if (x == std::numeric_limits<__int128>::min()) return o << "-170141183460469231731687303715884105728";
    if (x < 0) return o << "-" << -x;
    if (x < 10) return o << (char)(x + '0');
    return o << x / 10 << (char)(x % 10 + '0');
}

int main()
{
    std::string str = R"({"time": [0.143]})";
    std::cout << "input: " << str << std::endl;
    json j = json::parse(str);
    std::cout << "output: " << j.dump(4) << std::endl;
    double d = j["time"][0].get<double>();
    __int128_t d_128_bad = d * 10000000;
    __int128_t d_128_good = __int128(d * 1000) * 10000;
    std::cout << std::setprecision(16) << std::defaultfloat << d << std::endl;
    std::cout << "d_128_bad: " << d_128_bad << std::endl;
    std::cout << "d_128_good: " << d_128_good << std::endl;
}

Output:

input: {"time": [0.143]}
output: {
    "time": [
        0.143
    ]
}
0.143
d_128_bad: 1429999
d_128_good: 1430000

As you can see, the converted double is not the expected 1430000 instead it is 1429999. I know the reason is that a float point number can not be represented exactly. The problem can be solved if I know the number of digit after the decimal point.

For example,

I can instead use __int128_t(d * 1000) * 10000. However, I don't know the scale of a given number which might have a maximum of scale 7.

Question> Is there a possible solution for this? Also, I need to do this conversion very fast.

"a `__int128` with a maximum scale of 7(i.e. it may have 7 digits after the decimal point)" -- good news! An integer has no digits after the decimal point, so done! Did you misplace your "with" phrase? Perhaps you meant "a double with a maximum scale of 7 to a `__int128`"? Or maybe "convert, with a maximum scale of 7, to a `__int128`"? Or maybe... "convert a double, that had been converted from a decimal representation with a maximum scale of 7, to a `int128`"? — JaMiT, Apr 15 '22 at 03:53
You have not said which integer you want to convert a `double` to. From your text, it seems you might want to convert 1234.5 to 12345 and 54.375 to 54375. In other words, you want different scaling depending on the number of digits. But that raises the question of what you want for 123.50. Should that give 12350 or 1235? However, if some JSON had “123.50” and were parsed to a `double`, that double is a mathematical value 123.5; it is neither “123.5” nor “123.50”. So the information about its digits is gone. Please update the question to be clear about the result you want. — Eric Postpischil, Apr 15 '22 at 12:22
Note that the linked documentation is talking about a *decimal* datatype, rather than a `double` (binary) representation. — Bob__, Apr 15 '22 at 12:22
@Eric, Assume a double with a maximum scale of 7, then 1234.5 => 12345000000 and 54.375 => 543750000 and 0.1234567 => 1234567. — q0987, Apr 15 '22 at 13:01
@q0987 *"Assume a double with a maximum scale of 7, then [...] 0.1234567 => 1234567"* -- no, a **`double`** with maximum scale of 7 cannot store 0.1234567. A `double` is a binary representation of a number, and 0.1234567 has infinitely many digits in its binary representation. Also, "scale" for a `double` would refer to the number of *binary* digits in the representation, yet you are focused on decimal digits. It does look more and more like you mean a **number** (represented as a string of decimal digits) with a max scale of 7, which has been converted to an approximately equal `double`. — JaMiT, Apr 16 '22 at 01:23
More simply: You are using "scale" in the context of the decimal representation of a number, but "double" implies a binary representation. Mixing representations leads to confusion. — JaMiT, Apr 16 '22 at 01:30
@JaMiT, the double is received through a json format as `{"abc": 0.1234}`. If the number is represented as `{"abc": "0.1234"}`, then I could avoid all those issues. — q0987, Apr 16 '22 at 02:45
@q0987 JSON is a text-based protocol. It does not transmit `double`; it transmits numbers (in text format). The **number** is received through JSON as `0.1234`, not the `double`. The `double` is the transmitted number converted to a binary representation. The **number** has a maximum scale of 7; the `double` has no such limitation. — JaMiT, Apr 16 '22 at 06:56
@JaMiT, yes, I agree with what you say and am aware this issue already. The json parser I am using cannot return a text form for a double and it can only return a double if the value is represented as a number format. — q0987, Apr 16 '22 at 22:32
@q0987 The point is that converting "a double with a maximum scale of 7" to an integer is easy -- just multiply by 128 (a.k.a. 2 to the 7th). It appears what you want to express is the following: *The input to your system is processed by a library. One piece of input is a decimal number with a maximum scale of 7, which the library converts to a `double`. The goal is to obtain `10000000` times the original number, based on just the `double` (for performance). The naive approach of simple multiplication fails for `0.1430000` since this becomes `0.142999999999999988` as a `double`.* — JaMiT, Apr 17 '22 at 03:52
Since you always multiply by `10000000` regardless of the number's scale, and want to convert the result to integer: Does this answer your question? [Conversion from double to integer](https://stackoverflow.com/questions/24222701/conversion-from-double-to-integer) — JaMiT, Apr 17 '22 at 05:16

score 1 · Answer 1 · answered Apr 15 '22 at 12:16

1

I'm not familiar with this library, but it does appear to have a mechanism to get a json object's string representation (dump()). I would suggest you parse that into your value rather than going through the double intermediate representation, as in that case you will know the scale of the value as it was written.

answered Apr 15 '22 at 12:16

Pete Kirkham

48,893
5
92
171

In the real production data, the data is of an array with a very large number set. It is not applicable and expensive to call dump first and then do all those parsings manually. – q0987 Apr 15 '22 at 13:03

Double precision issues when converting it to a large integer

1 Answers1