3

I am not sure if this is possible at all in standard C++, so whether it even is possible to do, could be a secondary way to put my question.

I have this binary data which I want to read and re-create using structs. This data is originally created as a stream with the content appended to a buffer, field by field at a time; nothing special about that. I could simply read it as a stream, the same way it was written. Instead, I merely wanted to see if letting the compiler do the math for me, was possible, and instead implementing the binary data as a data structure instead.

The fields of the binary data have a predictable order which allows it to be represented as a data type, the issue I am having is with the depth and variable length of repeating fields. I am hoping the example code below makes it clearer.

Simple Example

struct Common {
    int length;
};

struct Boo {
    long member0;
    char member1;
};

struct FooSimple : Common {
    int count;
    Boo boo_list[];
};

char buffer[1024];
int index = 15;
((FooSimple *)buffer)->boo_list[index].member0;

Advanced Example

struct Common {
    int length;
};

struct Boo {
    long member0;
    char member1;
};

struct Goo {
    int count;
    Boo boo_list[];
};

struct FooAdvanced : Common {
    int count;
    Goo goo_list[];
};

char buffer[1024];
int index0 = 5, index1 = 15;
((FooAdvanced *)buffer)->goo_list[index0].boo_list[index1].member0;

The examples are not supposed to relate. I re-used some code due to lack of creativity for unique names.

For the simple example, there is nothing unusual about it. The Boo struct is of fixed size, therefore the compiler can do the calculations just fine, to reach the member0 field.

For the advanced example, as far as I can tell at least, it isn't as trivial of a case. The problem that I see, is that if I use the array selector operator to select a Goo object from the inline array of Goo-elements (goo_list), the compiler will not be able to do the offset calculations properly unless it makes some assumptions; possibly assuming that all preceding Goo-elements in the array have zero Boo-elements in the inline array (boo_list), or some other constant value. Naturally, that won't be the case.

Question(s):

  1. What ways are there to achieve the offset computations to be done by the compiler, despite the inline arrays having variable lengths? Unless I am missing something, I believe templates can't help at all, due to their compile-time nature.
  2. Is this even possible to achieve in C++?
  3. How do you handle the case with instantiating a FoodAdvanced object, by feeding a variable number of Goo and Boo element counts to the goo_list and boo_list members, respectively?
  4. If it is impossible, would I have to write some sort of wrapper code to handle the calculations instead?
Hello World
  • 423
  • 5
  • 15
  • 3
    If you want to serialize data you should take a look at boost serialization. – NathanOliver May 30 '19 at 17:58
  • @NathanOliver or myriad of other serialization solutions out there. The question is unclear. – SergeyA May 30 '19 at 18:02
  • 3
    You should read about serialization. In most cases, it's nowhere near as simple as reading an object's byte representation. To handle things like pointers, dynamic arrays, polymorphic types, graph data structures, or even endianness, you'll need a good serialization system. Some good ones exist already, like boost serialization, google protocol buffers, as well as human-readable plain text encodings like JSON/XML/YAML – alter_igel May 30 '19 at 18:04
  • I forgot to mention this in the question, but of course you may freely place questions for further clarification. – Hello World May 30 '19 at 18:04
  • 2
    @HelloWorld rest assured that this is indeed an interesting question, and certainly one that is commonly asked and useful to answer. It's simply unfortunate that, being a complicated issue in C++ specifically, it's difficult to answer directly in StackOverflow's Q/A format. Otherwise, one could answer by recommending third party libraries, but that is specifically off-topic on StackOverflow. See this question also: https://stackoverflow.com/questions/234724/is-it-possible-to-serialize-and-deserialize-a-class-in-c – alter_igel May 30 '19 at 18:08
  • @alterigel OP never mentions serialization. It was assumed by Nathan that this is what they are asking for, but OP never took time to clarify the question. I maintain the question is unclear and doesn't warrant an answer. – SergeyA May 30 '19 at 18:15
  • 1
    @NathanOliver `boost::serialization` works well with uniform endpoints (code written using `boost::serialization`). Json, Google Protobuf whatever would be a better choice to use in a variety of endpoint implementations. – πάντα ῥεῖ May 30 '19 at 18:15
  • @SergeyA I presume I'm a newbie to the SO standards for question format. I have provided with context and elaboration on the issue, which makes up for most of my question, and finally four questions that should relate to the context. I will gladly add to the question, if you are willing to provide feedback. – Hello World May 30 '19 at 18:21
  • 1
    @HelloWorld is [this kind of variable-length struct](https://stackoverflow.com/questions/7641698/allocating-struct-with-variable-length-array-member) what you're trying to achieve? I'm afraid this simply isn't done in safe/standard C++. The usual approach would be something like `struct Goo { std::vector boo_list; };` – alter_igel May 30 '19 at 18:55
  • @alterigel That is directly related to what I am after. That would be the "Simple Example" as I demonstrate above. It might be helpful to know that it is known as the "struct hack," although I guess the "struct hack" gets a little challenging when it comes to a more complex data structure, as demonstrated in the "Advanced Example." Thank you for all references to information. It is all being useful to me. – Hello World May 30 '19 at 19:33
  • 1
    @HelloWorld I'm glad I could help. I believe [Google Protocol Buffers](https://developers.google.com/protocol-buffers/) would be the closest tool that does something like what you're trying to achieve. It isn't necessarily the best tool that exists, but it allows you to interact with structured binary data in a clean-ish way. It does impose a lot of restrictions on you. For example, it generates class definitions for you, and I don't think the binary format is very customizable. I'm not experienced with it though. You'd have to read the docs to be sure. – alter_igel May 30 '19 at 19:45

0 Answers0