Read binary data to long int

Question

I need to read binary data which contain a column of numbers (time tags) and use 8bytes to record each number. I know that they are recorded in little endian order. If read correctly they should be decoded as (example)

I recognize that the above numbers are on the 2^31 -1 threshold. I try to read the data and invert the endiandness with: (length is the total number of bytes and buffer is pointer to an array that contains the bytes)

unsigned long int tag;
//uint64_t tag;    
for (int j=0; j<length; j=j+8) //read the whole file in 8-byte blocks
   { tag = 0;  
     for (int i=0; i<=7; i++) //read each block ,byte by byte
        {tag ^=  ((unsigned char)buffer[j+i])<<8*i ;} //shift each byte to invert endiandness and add them with ^=
   }
                                                                                              }

when run, the code gives:

  ...  
  2147426467  
  2147426635  
  18446744071562097256  
  similar big numbers   
  ...

The last number is not (2^64 - 1 - correct value). Same result using uint64_t tag. The code succeeds with declaring tag as

unsigned int tag;

but fails for tags greater than 2^32 -1. At least this makes sense.
I suppose I need some kind of casting on buffer[i+j] but I don't know how to do it.

(static_cast<uint64_t>(buffer[j+i]))

also doesn't work.
I read a similar question but still need some help.

@jxh how can I promote it to unsigned long instead of signed int ? — Lefteris, Mar 28 '18 at 18:29
@user2079303 it is declared as char `char *buffer = new char[length];` — Lefteris, Mar 28 '18 at 18:45
Related question: https://stackoverflow.com/q/15911714/315052 — jxh, Mar 28 '18 at 18:48

Koto · Answer 1 · 2018-03-28T18:43:50.210

1

You use a temporary value. The computer will automatically reserve the least amount needed to store a temporary value. In your case that would be the 32 bits. Once you shift the byte further than 32 bits it will be shifted into oblivion. In order to fix this you need to explicitly store the value in a 64 bit integer first. So instead of

    {tag ^=  ((unsigned char)buffer[j+i])<<8*i ;}

you should use something like this

    {
       unsigned long long tmp = (unsigned char)buffer[j+i];
       tmp <<= 8*i;
       tag ^=  tmp;
    }

edited Mar 28 '18 at 18:43

answered Mar 28 '18 at 18:36

Koto

506
2
4
15

works perfectly. It works also with `unsigned long tmp` (at least on my machine) – Lefteris Mar 28 '18 at 18:59
long might not be 64 bits on all systems. See https://stackoverflow.com/questions/6462439/whats-the-difference-between-long-long-and-long for more information. – Koto Mar 28 '18 at 19:10

jxh · Accepted Answer · 2018-03-28T19:13:25.050

We assume that buffer[j+i] is a char, and that chars are signed on your platform. Casting to unsigned char converts buffer[j+i] into an unsigned type. However, when applying the << operator, the unsigned char value gets promoted to int so long as an int can hold all values representable by unsigned char.

Your attempt to cast buffer[j+i] directly to uint64_t fails because if char is signed, the sign extension is still applied before the value is converted to the unsigned type.

A double cast may work (that is, cast to unsigned char and then to unsigned long), but using an unsigned long variable to hold the intermediate value should make the intention of the code more clear. For me, the code would look like:

decltype(tag) val = static_cast<unsigned char>(buffer[j+i]);
tag ^= val << 8*i;

works perfectly. thank you for the explanation. I wish I could accept both answers. — Lefteris, Mar 28 '18 at 19:04
If you make `buffer` contain `unsigned char` to being with, you can get rid of casting when assigning to `val`. — jxh, Mar 28 '18 at 19:32

Read binary data to long int

2 Answers2