how to convert C structure into TLV format or flat memory buffer

Question

I need to convert one C structure into TLV format and send it across. Can someone help me with same? My structure is a really nested structure with lot of in-directions (pointers) to different structures and lot of void *. So, i am stuck and could use help.

struct a {    
    char a[100];    
    char b;    
    struct b *tag;    
    struct c *value;    
    void *data;    
};

struct b {    
    void *data;    
    int a;    
};

struct c {    
    void *data;    
    char arr[100];    
};

1) Why does your tag have two parts to it? 2) why are you using `b` as both the structure name and a char within `a` 3) which part of this do you want in TLV format? (just `a` or are there sub TLV's within `a`)? — Mike, Aug 06 '13 at 16:41
Hi Mike, it was just an example. Agreed i choose duplicate name for both a and b. I want to encode and flatten complete struct a. — puzzled confused, Aug 06 '13 at 16:47
This isn't that hard. Remove all the indirections. Why even have separate struct bs and cs if you're going to serialize them anyways? — KrisSodroski, Aug 06 '13 at 16:57
i cant remove all indirections...as thats the parent structure which i want to send across. i was just giving idea of kind of nesting i am having in my code. — puzzled confused, Aug 06 '13 at 17:08
Yes, basic idea is i want o serialize structure a which has nested indirections to struct b and struct c. So, in turn i want to flatten complete structure a [this would mean flattening struct b and struct c as well] — puzzled confused, Aug 06 '13 at 17:18

score 6 · Accepted Answer · answered Aug 06 '13 at 17:33

TLV stands for Tag Length Value, and that's exactly what it is. You might have some confusion on this point. Normal usage of this is when you get a large buffer of bytes, perhaps being filled from some serial device, and you need to know what is in that buffer.

Take for example the communication between a SIM card and a phone. There are a number of well defined Tags each corresponding to a type of message that will be decoded. (These are all defined in specification such as ISO 7816-4) and for example if you want to write to a binary file in the SIM card you need to tell the SIM card how many bytes of data you want to send it. So you'd construct some message like:

   A0 D0 00 00 0A 01 02 03 04 05 06 07 08 09 0A 
  +-----+     +-+ +---------------------------+
     |         |              |
    tag     length          Value

// A0D0 - the tag tells the SIM I want to "write a binary file to a GSM SIM card"
// 0A -   Says the aforementioned write will be 10 bytes in Length
// 0102... - Then the Value follows the length

So in this case we used TLV to send a buffer full of bytes, the receiving device (SIM card in this case) will parse the Tag, known to expect the Length then know how many bytes of "Value" are coming before the transmission is complete. Note this isn't a full true TLV because each piece of data doesn't have it's own TL some are just known (such as those 00's between the "tag" and "length", those are parameters and they're set to be 1 byte and always follow the instruction is it's not needed to have a tag or length)

So that's the overview. Now where does that leave us with your issue? First, as I hope you can see now, we need to know what is going to be Tagged. That depends on who is expecting the data, this is something you should know. Looking at your problem I'd think it was something like this:

Client A generates a "struct a" to send to client B. (There must be a "struct a" tag)
"struct a" is comprised of "struct b" and "struct c" so we need tags of these as well

In order of Client B to be able to read these values we need to define the Tags:

// Tags for structures
#define TAG_A 0x90       // These values are made up but it's important to note
#define TAG_B 0x91       // both Client A and Client B must know what the tags mean
#define TAG_C 0x92       // what what value they are set to

Ideally since you have embedded data within each structure you would have sub tags as well:

// Tags for struct A:
#define TAG_A_FIX_DATA 0x93
#define TAG_A_VAR_DATA 0x94
#define TAG_B_FIX_DATA 0x95
#define TAG_B_VAR_DATA 0x96

So each of your structs would be filled with data as normal then when you go to send the data you'd deconstruct the values into a buffer. The following pseudo code gives you the idea

unsigned char *buffer = malloc(/*big enough for struct a+b+c+tags*/);
buffer[0] = TAG_A;
buffer[1] = /*size of your A structure*/
buffer[2] = TAG_A_FIX_DATA;
buffer[3] = 101; // for the array and the char.. if that's how you want to handle it
buffer[4-105] = a.a and a.b;
buffer[106] = TAG_B;
buffer[107] = /*length of struct B*/
...

So when Client B gets the large buffer of data they can construct their own local struct a, struct b, and struct c then parse out the fields and populate.

Thanks Mike for the great details. That really made my life easier. One doubt still? How to handle void *data and how to handle/encode the void * data embedded in the structure you want to send in TLV format in a scenario where you are not sure of the length of the void * data. — puzzled confused, Aug 06 '13 at 18:21
@puzzle - T*L*V, as the name implies you must know the length of the data. If you are getting this structure from an outside source they need to include a field (another `int` or something) to let you know the length. The only other exception to that would be if there is a terminator to the `void * data` for example when you saw the bytes `0xDEAD` you know the data is done, then you can calculate the length yourself. — Mike, Aug 06 '13 at 18:24

how to convert C structure into TLV format or flat memory buffer

1 Answers1