(de)serialize struct containing __uint128_t in c++

Question

I am trying to serialize and deserialize a vector of structs containing the __uint128_t datatype. Following is an attempt to write a simplified version, without utilizing sizeof(Test) to serialize:

typedef __uint128_t data_t;
//typedef uint64_t data_t;

struct Test {
data_t element;
int count;

Test()=default;

};

template<typename POD>
std::ostream& serialize(std::ostream& os, const POD& v)
{
  static_assert(std::is_trivial<POD>::value && std::is_standard_layout<POD>::value,
    "Can only serialize POD types with this function");

  printf("in POD, sizeof(POD): %lu\n", sizeof(v));
  os.write(reinterpret_cast<const char*>(&v), sizeof(POD));
  return os;
}

template<typename POD>
std::istream& deserialize(std::istream& is, POD& v)
{
  static_assert(std::is_trivial<POD>::value && std::is_standard_layout<POD>::value,
    "Can only deserialize POD types with this function");

  is.read(reinterpret_cast<char*>(&v), sizeof(POD));
  return is;
}

int main()
{

 std::string tmp_buffer;
 std::stringstream oss;
 
 std::vector<Test>data_in(2);

 data_in[0].element=2345;
 data_in[0].count=4;

 data_in[1].element=5431;
 data_in[1].count=10;

 serialize(oss, data_in[0].element);
 serialize(oss, data_in[0].count);
 serialize(oss, data_in[1].element);
 serialize(oss, data_in[1].count);

 tmp_buffer = oss.str();
 printf("buffer len: %lu\n", tmp_buffer.size());

 Test k1;
 deserialize(oss, k1.element);
 deserialize(oss, k1.count);
 printf("data_out0: element: %llu, count: %d\n", k1.element, k1.count);

 Test k2;
 deserialize<data_t>(oss, k2.element);
 deserialize<int>(oss, k2.count);
 printf("data_out1: element: %llu, count: %d\n", k2.element, k2.count);

 assert(data_in[0].element == k1.element);
 assert(data_in[0].count == k1.count); /* passes ? */

 int c=k1.count;
 Test x = Test{k1.element, c};
 printf("c: %d, element: %llu, count: %d\n", c, x.element, x.count);

 return 0;

}

The output is as follows:

in POD, sizeof(POD): 16
in POD, sizeof(POD): 4
in POD, sizeof(POD): 16
in POD, sizeof(POD): 4
buffer len: 64
data_out0: element: 2345, count: 0
data_out1: element: 5431, count: 0
c: 4, element: 2345, count: 0

The value of count is printed as 0, even though it passes the assert statement. Why am I seeing correct values for element and not for count?

If data_t is set as uint64_t, the output is correct for values of both element and count. Why am I seeing a discrepancy for the __uint128_t case?

score 2 · Accepted Answer · answered Jan 15 '21 at 00:50

The assert passes because its condition is true. The problem is actually that you're trying to print x.element before x.count. x.element is an extended integer type, and there is no support for printing extended integers in the C standard library (extended integers don't exist in the C standard), so printf doesn't know how to handle the data you're giving it. There is no guarantee as to what printf will do in such a case; it's implementation-dependent. Not to mention, you're trying to print it with %llu, implying that it's an unsigned long long, but it isn't. An unsigned long long is guaranteed to have at least 64 bytes of space; your extended integer type requires 128 bytes of space.

If you don't print x.element, then x.count will print just fine.

Thank you for pointing this out. I am glad to know that the problem was not in my approach to (de)serialize. — PGOnTheGo, Jan 15 '21 at 01:03

score 2 · Answer 2 · answered Jan 15 '21 at 00:50

2

Instead of

printf("data_out1: element: %llu, count: %d\n", k2.element, k2.count);

try to print count only:

printf("data_out1: count: %d\n", k2.count);

It should print OK.

The problem is that printf's %llu doesn't work with 128-bit value

answered Jan 15 '21 at 00:50

Vlad Feinstein

10,960
1
12
27

Thank you Vlad!! That worked. I am glad that the original approach is sound. – PGOnTheGo Jan 15 '21 at 01:06

(de)serialize struct containing __uint128_t in c++

2 Answers2