4

Consider the following code:

#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
    long double test = 0xFFFFFFFFFFFFFFFF;
    cout << "1: " << test << endl;
    unsigned long long test2 = test;
    cout << "2: " << test2 << endl;
    cout << "3: " << (unsigned long long)test << endl;
    return 0;
}

Compiling this code with GCC g++ (7.5.0) and running produces the following output as expected:

1: 1.84467e+19
2: 18446744073709551615
3: 18446744073709551615

However compiling this with the Microsoft Visual C++ compiler (16.8.31019.35, both 64-bit and 32-bit) and running produces the following output:

1: 1.84467e+19
2: 9223372036854775808
3: 9223372036854775808

When casting a value to an unsigned long long, the MSVC compiler won't give a value lager than the max of a (signed) long long.

Am I doing something wrong? 

Am I running into a compiler limitation that I do not know about?

Does anyone know of a possible workaround to this problem?

ManInBlack
  • 103
  • 6
  • 1
    what is "the problem" you need a workaround for? If you do not cast there is no problem. – 463035818_is_not_an_ai Mar 04 '22 at 11:44
  • @463035818_is_not_a_number I want to take a long double that contains a value greater than 0x8000000000000000 and convert in to a unsigned long long that contains the same values. The printing of the values is just to demonstrate the issue. – ManInBlack Mar 04 '22 at 11:47
  • 1
    In msvc a long double is the same as double and is 8 bytes not 16. – drescherjm Mar 04 '22 at 11:48
  • 1
    try to take a look at `std::numeric_limits::max()` and relatives. This might shed some light on it. What do you want to do when the value of the `long double` cannot fit in a `unsigned long long` ? – 463035818_is_not_an_ai Mar 04 '22 at 11:49
  • @drescherjm, thanks for pointing this out. I was not aware of it. – ManInBlack Mar 04 '22 at 11:56
  • @463035818_is_not_a_number, I am not sure I understand. `std::numeric_limits::max()` is equal to 0xFFFFFFFFFFFFFFFF, so it should fit just fine. I am handling the case where it cannot fit separately. – ManInBlack Mar 04 '22 at 11:58
  • yeah sorry I was looking at the wrong thing. – 463035818_is_not_an_ai Mar 04 '22 at 11:59
  • 2
    Ok, I think I know what is going on. Because a MSVC `long double` is really just a `double`, it does not have enough precision to contain the exact value of 0xFFFFFFFFFFFFFFFF. When this value is stored in the `long double` it gets "rounded" to a value that is lager than 0xFFFFFFFFFFFFFFFF. This then causes undefined behaviour when converting to an `unsigned long long`. Thanks for the help. – ManInBlack Mar 04 '22 at 12:04
  • 1
    @ManInBlack Note that it is perfectly fine to answer your question yourself and that comment looks like it should be an answer. – user17732522 Mar 04 '22 at 12:07
  • @drescherjm gcc doesn't emulate anything, it's using the 80-bit extended precision type in x86 and pad to 12 or 16 bytes for alignment. [Why did Microsoft abandon long double data type?](https://stackoverflow.com/q/7120710/995714) – phuclv Mar 04 '22 at 12:52
  • Does this answer your question? [Why are double and long double completely the same on my 64 bit machine?](https://stackoverflow.com/questions/8922216/why-are-double-and-long-double-completely-the-same-on-my-64-bit-machine) – phuclv Mar 04 '22 at 12:53
  • @phuclv thanks. I didn't know how it accomplished this given the CPU does not have full quad double. I see there is info about that here: [https://en.wikipedia.org/wiki/Extended_precision](https://en.wikipedia.org/wiki/Extended_precision) – drescherjm Mar 04 '22 at 12:54

2 Answers2

6

Because a MSVC long double is really just a double (as pointed out by @drescherjm in the comments), it does not have enough precision to contain the exact value of 0xFFFFFFFFFFFFFFFF. When this value is stored in the long double it gets "rounded" to a value that is lager than 0xFFFFFFFFFFFFFFFF. This then causes undefined behaviour when converting to an unsigned long long.

ManInBlack
  • 103
  • 6
2

You are seeing undefined behaviour because, as pointed out in the comments, a long double is the same as a double in MSVC and the 'converted' value of your 0xFFFFFFFFFFFFFFFF (or ULLONG_MAX) actually gets 'rounded' to a slightly (but significantly) larger value, as can be seen in the following code:

int main(int argc, char* argv[])
{
    long double test = 0xFFFFFFFFFFFFFFFF;
    cout << 0xFFFFFFFFFFFFFFFFuLL << endl;
    cout << fixed << setprecision(16) << endl;
    cout << test << endl;
    return 0;
}

Output:

18446744073709551615
18446744073709551616.0000000000000000

Thus, when converting that floating-point value back to an unsigned long long, you are falling foul of the conversion rules specified in this Microsoft document:

  • For conversion to unsigned long or unsigned long long, the result of converting an out-of-range value may be some value other than the highest or lowest representable value. Whether the result is a sentinel or saturated value or not depends on the compiler options and target architecture. Future compiler releases may return a saturated or sentinel value instead.

This UB can be further 'verified' (for want of a better term) by switching to the clang-cl compiler that can be used from within Visual Studio. For your original code, this then gives 0 for the values on both the "2" and "3" output lines.

Assuming that the clang (LLVM) compiler is not bound by the aforementioned "Microsoft Rules," we can, instead, fall back on the C++ Standard:

7.10 Floating-integral conversions      [conv.fpint]

1     A prvalue of a floating-point type can be converted to a prvalue of an integer type. The conversion truncates; that is, the fractional part is discarded. The behavior is undefined if the truncated value cannot be represented in the destination type.

Adrian Mole
  • 49,934
  • 160
  • 51
  • 83