-2

I would like to shift 0xff left by 3 bytes and store it in a uint64_t, which should work as such:

uint64_t temp = 0xff << 24;

This yields a value of 0xffffffffff000000 which is most definitely not the expected 0xff000000.

However, if I shift it by fewer than 3 bytes, it results in the correct answer.

Furthermore, trying to shift 0x01 left by 3 bytes does work.

Here's my output:

0xff shifted by 0 bytes: 0xff
0x01 shifted by 0 bytes: 0x1
0xff shifted by 1 bytes: 0xff00
0x01 shifted by 1 bytes: 0x100
0xff shifted by 2 bytes: 0xff0000
0x01 shifted by 2 bytes: 0x10000
0xff shifted by 3 bytes: 0xffffffffff000000
0x01 shifted by 3 bytes: 0x1000000

With some experimentation, left shifting works up to 3 bits for each uint64_t up to 0x7f, which yields 0x7f000000. 0x80 yields 0xffffffff80000000.

Does anyone have an explanation for this bizarre behavior? 0xff000000 certainly falls within the 264 - 1 limits of uint64_t.

phuclv
  • 37,963
  • 15
  • 156
  • 475
Josh Mathews
  • 198
  • 1
  • 15
  • 2
    `0xff` is an `int`, which is usually a 32-bit integer. So the `0xff << 24` is doing the math on 32 bit integers, not on `uint64_t`s. So the leftmost `1` bit is being treated as the sign bit, so when we extend the width to a 64-bit integer, we set all new bits to `1`, which makes all the `f`s when we convert to an unsigned integer – Justin Sep 15 '17 at 19:36
  • "bizarre" as in "I don't understand it"? The reason for that behavior is that all those constants have type `int`. To get the behavior that you're looking for, make sure that they are 64 bits wide, either by explicitly writing them as unsigned long long values (e.g., 0xffULL) or by casting to `std::uint64_t`. – Pete Becker Sep 15 '17 at 19:36
  • 1
    Try `uint64_t temp = 0xffu << 24;` or `uint64_t temp = 0xfful << 24;` – simon Sep 15 '17 at 19:37
  • 1
    @gurka Better to suggest `uint64_t(0xff) << 24` as `long` doesn't have to be 64 bits wide. – NathanOliver Sep 15 '17 at 19:38
  • @gurka -- `unsigned` is only required to be 16 bits wide, and `unsigned long` 32 bits wide. They **might** be wider with your compiler, but to ensure 64 bits, use 64 bit types, either `unsigned long long` (which is at least 64 bits) or `std::uint64_t` which, if it exists, is exactly 64 bits wide. – Pete Becker Sep 15 '17 at 19:39
  • @PeteBecker I know, that's why I wrote _Try_ and not _Use_. Comparing what happens when you leftshift `0xff`, `0xffu`, `0xfful` and `0xffull` might be a good way to understand the behaviour. – simon Sep 15 '17 at 19:43
  • Thank you, Justin! I appreciate your help! Just not paying attention to the details: constants are ints, not uint64_t. – Josh Mathews Sep 15 '17 at 19:47
  • Possible duplicate of [Is unsigned long int correct for this operation?](https://stackoverflow.com/questions/35119136/is-unsigned-long-int-correct-for-this-operation) – phuclv Oct 16 '18 at 10:19
  • tons of other duplicates: [bit shifting with unsigned long type produces wrong results](https://stackoverflow.com/q/31744305/995714), [Unsigned long long overflow error?](https://stackoverflow.com/q/23976201/995714)... – phuclv Oct 16 '18 at 10:20

4 Answers4

2

Does anyone have an explanation for this bizarre behavior?

Yes, type of operation always depend on operand types and never on result type:

double r = 1.0 / 2.0; 
    // double divided by double and result double assigned to r
    // r == 0.5

double r = 1.0 / 2; 
    // 2 converted to double, double divided by double and result double assigned to r
    // r == 0.5

double r = 1 / 2; 
    // int divided by int, result int converted to double and assigned to r
    // r == 0.0

When you understand and remenber this you would not fall on this mistake again.

Slava
  • 43,454
  • 1
  • 47
  • 90
1

I suspect the behavior is compiler dependent, but I am seeing the same thing.

The fix is simple. Be sure to cast the 0xff to a uint64_t type BEFORE performing the shift. That way the compiler will handle it as the correct type.

uint64_t temp = uint64_t(0xff) << 24
MikeMayer67
  • 570
  • 6
  • 17
  • it's not compiler dependent, because 0xFF is always a literal of type `int`, and that behavior always happen if int is a 32-bit type. You just need `0xFFULL` or `UINT64_C(0xFF)` instead of a cast like that. There are already a lot of duplicates on this: [Bitwise shift operation in C on uint64_t variable](https://stackoverflow.com/q/9290823/995714), [Calculating maximum int value in C program (1 << 31) - 1](https://stackoverflow.com/q/17019187/995714), [Weird result after assigning 2^31 to a signed and unsigned 32-bit integer variable](https://stackoverflow.com/q/9973344/995714) – phuclv Oct 16 '18 at 10:18
  • [Type of integer literals not int by default?](https://stackoverflow.com/q/8108642/995714), [What is the default type of integral literals represented in hex or octal in C++?](https://stackoverflow.com/q/38782709/995714) – phuclv Oct 16 '18 at 10:26
0

Shifting left creates a negative (32 bit) number which then gets filled to 64 bits.

Try 

0xff << 24LL;
mksteve
  • 12,614
  • 3
  • 28
  • 50
  • 1
    The type of the right operand to a shift doesn't affect the type of the result; using `24LL` is no different from using `24` here. The shift operators are different from all the other operators in this respect. For the other arithmetic and logical operators, the smaller type is promoted to match the larger one. For shift operators the only argument that might be promoted is the left-hand one, and that's only if it's `char`, `short`, or their unsigned variants. – Pete Becker Sep 15 '17 at 22:26
0

Let's break your problem up into two pieces. The first is the shift operation, and the other is the conversion to uint64_t.

As far as the left shift is concerned, you are invoking undefined behavior on 32-bit (or smaller) architectures. As others have mentioned, the operands are int. A 32-bit int with the given value would be 0x000000ff. Note that this is a signed number, so the left-most bit is the sign. According to the standard, if you the shift affects the sign-bit, the result is undefined. It is up to the whims of the implementation, it is subject to change at any point, and it can even be completely optimized away if the compiler recognizes it at compile-time. The latter is not realistic, but it is actually permitted. While you should never rely on code of this form, this is actually not the root of the behavior that puzzled you.

Now, for the second part. The undefined outcome of the left shift operation has to be converted to a uint64_t. The standard states for signed to unsigned integral conversions:

If the destination type is unsigned, the resulting value is the smallest unsigned value equal to the source value modulo 2n where n is the number of bits used to represent the destination type.

That is, depending on whether the destination type is wider or narrower, signed integers are sign-extended[footnote 1] or truncated and unsigned integers are zero-extended or truncated respectively.

The footnote clarifies that sign-extension is true only for two's-complement representation which is used on every platform with a C++ compiler currently.

Sign-extension means just that everything left of the sign bit on the destination variable will be filled with the sign-bit, which produces all the fs in your result. As you noted, you could left shift 0x7f by 3-bytes without this occurring, That's because 0x7f=0b01111111. After the shift, you get 0x7f000000 which is the largest signed int, ie the largest number that doesn't affect the sign bit. Therefore, in the conversion, a 0 was extended.

Converting the left operand to a large enough type solves this.

uint64_t temp = uint64_t(0xff) << 24
Community
  • 1
  • 1
patatahooligan
  • 3,111
  • 1
  • 18
  • 27
  • 1
    `converting one of the two arguments before the shift operation solves this` this isn't correct. Shift operators are special and you must cast the first argument instead of any of them like other operators – phuclv Oct 16 '18 at 10:21
  • @phuclv I've never come across this before, but it seems you are right. – patatahooligan Nov 04 '18 at 11:26