Serialization of struct

Question

Suppose i have a struct whose member values i want to send over the network to another system using winsock 2. I'm using C++ language. How do i convert it to char * keeping in mind that the struct has to be serialized before sending and also how do i deserialize the char * into struct at the other end? I found boost serialization as a suggestion to similar question but can anyone illustrate with a small code snippet for both serialization and deserialization ?

This question might seem very basic but the other answers to the related posts did not help much.

What about [serialization example](http://www.boost.org/doc/libs/1_53_0/libs/serialization/example/demo.cpp) on the Boost web-site? — nogard, May 14 '13 at 12:59

score 30 · Answer 1 · answered Dec 30 '13 at 10:50

30

Following example shows a simplest way to serialize struct into char array and de-serialize it.

#include <iostream>
#include <cstring>

#define BUFSIZE 512
#define PACKETSIZE sizeof(MSG)

using namespace std;

typedef struct MSG
{
    int type;
    int priority;
    int sender;
    char message[BUFSIZE];
}MSG;

void serialize(MSG* msgPacket, char *data);
void deserialize(char *data, MSG* msgPacket);
void printMsg(MSG* msgPacket);

int main()
{
    MSG* newMsg = new MSG;
    newMsg->type = 1;
    newMsg->priority = 9;
    newMsg->sender = 2;
    strcpy(newMsg->message, "hello from server\0");
    printMsg(newMsg);

    char data[PACKETSIZE];

    serialize(newMsg, data);

    MSG* temp = new MSG;
    deserialize(data, temp);
    printMsg(temp);

    return 0;
}

void serialize(MSG* msgPacket, char *data)
{
    int *q = (int*)data;    
    *q = msgPacket->type;       q++;    
    *q = msgPacket->priority;   q++;    
    *q = msgPacket->sender;     q++;

    char *p = (char*)q;
    int i = 0;
    while (i < BUFSIZE)
    {
        *p = msgPacket->message[i];
        p++;
        i++;
    }
}

void deserialize(char *data, MSG* msgPacket)
{
    int *q = (int*)data;    
    msgPacket->type = *q;       q++;    
    msgPacket->priority = *q;   q++;    
    msgPacket->sender = *q;     q++;

    char *p = (char*)q;
    int i = 0;
    while (i < BUFSIZE)
    {
        msgPacket->message[i] = *p;
        p++;
        i++;
    }
}

void printMsg(MSG* msgPacket)
{
    cout << msgPacket->type << endl;
    cout << msgPacket->priority << endl;
    cout << msgPacket->sender << endl;
    cout << msgPacket->message << endl;
}

answered Dec 30 '13 at 10:50

Amith Chinthaka

1,015
1
17
24

2

Great example of a simple serialization – PhillyNJ Jul 12 '15 at 00:15
1

Your code has two memory leaks. Actually there is no need to use the `new` at all. Here are further possible improvements: 1. make the MSG a return type of the deserialize function instead of an output parameter 2. instead of `typedef struct MSG{...}MSG;` use `struct MSG{...};`, it's the same in C++ 3. use `std::string` or `std::vector` to hold the serialized data (and dispose of that `#define PACKETSIZE` altogether) and use it as a return value of the `serialize` function instead of the output parameter (or if you insist change the parameter to `char(&data)[PACKETSIZE]`). – Marian Spanik Feb 02 '16 at 17:46
There are no memory leaks in his code. All memory is freed at the end of the main method. – jackmott May 17 '16 at 20:26
Doesn't his break the strict aliasing rule when dereferencing `q` in `serialize`? – Victor Savu Dec 12 '16 at 09:30
Yes, this is completely invalid. – Lightness Races in Orbit Jul 23 '19 at 15:44
2

This assumes that the sender and receiver are aware of the members of the struct. If details of each member of the struct is also embedded and sent, the receiver could use this to frame appropriate pointers. A simple change in one end without the other end knowing about it will cause a lot of bugs – AlphaGoku Jan 21 '20 at 06:18
3

This is a good simple implementation, true. But, there are cases where this implementation would definitely not work: 1) if the sender and receiver do not have the same struct implementation (as @AlphaGoku pointed out), 2) if the compiler or compiler flags are different (e.g. one compiler decides to remove the padding bytes between the struct elements, or, if `int` is 4 bytes in one compiler and 2 bytes in the other), or, 3) even more insidious, if the sender and receiver do not have the same endianness (e.g. the sender is little-endian and the receiver is big-endian). – adentinger May 14 '20 at 15:29
@AnthonyD973 1) could not matter if the versioning is done at protocol level. 2) padding doesn't matter cause struct members are referred directly, only the buffer is advanced. size of ints can be taken into account using types like int32_t or int64_t from 3) can be fixed using endianness functions like boost big_to_native_inplace or endianness macro so buffer is always big endian order for example – AndrewBloom Feb 16 '21 at 13:29
1

@AndrewBloom I agree with you these problems *can* be worked around (otherwise different machines wouldn't be able to communicate over a network, a bus or otherwise!). What I meant was that this answer only gives the code and not the logic behind it. As the code currently is, it suffers from the problems I explained. – adentinger Feb 17 '21 at 14:06

rwols · Answer 2 · 2013-05-14T13:28:23.187

You can just do

struct MyStruct {

    int data;
    char* someNullTerminatedName; // Assuming not larger than 1023 chars

    std::ostream& serialize(std::ostream& os) const {
        char null = '\0';
        os.write((char*)&data, sizeof(data));
        os.write(someNullTerminatedName, strlen(someNullTerminatedName));
        os.write(&null, 1);
        return os;
    }
    std::istream& deserialize(std::istream& is) {
        char buffer[1024];
        int i = 0;
        is.read((char*)&data, sizeof(data));
        do { buffer[i] = is.get(); ++i; } while(buffer[i] != '\0');
        if (someNullTerminatedName != NULL) free(someNullTerminatedName);
        someNullTerminatedName = (char*)malloc(i);
        for (i = 0; buffer[i] != '\0'; ++i) {
            someNullTerminatedName[i] = buffer[i];
        }
        return is;
    }
};

It's up to you to take care of endianness and differences in size of ints and whatnot.

Example:

MyStruct foo, bar;
std::stringstream stream;
foo.serialize(stream);
// ... Now stream.str().c_str() contains a char* buffer representation of foo.
// For example it might contain [ 1f 3a 4d 10 h e l l o w o r l d \0 ]
bar.deserialize(stream);
// ... Now bar is a copy, via a serial stream of data, of foo.

If you have a socket library that exposes its interface via C++ iostreams then you don't even need the stringstream.

score 6 · Answer 3 · answered May 14 '13 at 13:28

You can also have a look at Protocol Buffers from Google which is a platform/language independent library for sending data between hosts.

However, the paradigm is shifted towards writing the protocol first and then fitting your data structures into it. The advantage of this though is that it forces your software architecture to fit well with simple data types.

score 3 · Answer 4 · answered May 14 '13 at 12:59

3

If your struct is POD you can use memcpy:

::memcpy(data, &your_struct, sizeof(YourStruct)));

and vice versa at reception:

::memcpy(&your_struct, data, sizeof(YourStruct)));

Where data is a char*. Don't forget you have to Allocate it, making sure it's big enought and delete it at the end.

answered May 14 '13 at 12:59

0x26res

11,925
11
54
108

1

At the sending end:struct example{ int a; float b; double c; }; example ex; ex.a = 1; ex.b = 1.02; ex.c = 1.040; ::memcpy(Buffer, &ex, sizeof(example)); retval = send(conn_socket, Buffer, sizeof(Buffer), 0); And at the receiving end: example ex; retval = recvfrom(msgsock,Buffer, sizeof(Buffer), 0, (struct sockaddr *)&from, &fromlen); ::memcpy(&ex, Buffer, sizeof(example)); printf("%f", ex.c); The output: -62774385622041925000000000000000000000000000000. Why is this? – Vigo May 14 '13 at 13:19
9

You need to be very careful when just mem-copying structs because different architectures and different compilers will apply padding and endianness differently. – Nick May 14 '13 at 13:27
I am using Visual studio 2005 and XP on both systems. But why this difference? – Vigo May 14 '13 at 13:38
where does your buffer come from ? Are you sure it's bigger than your struct ? – 0x26res May 14 '13 at 13:48
yes.. i have malloc-ed 60 bytes for my buffer whereas the the struct is easily less than 60.. – Vigo May 14 '13 at 13:55
ok but if you malloc 60 bytes in your buffer and your buffer is a char* do you know what, I'm concern that sizeof(buffer) is 4 bytes / 8 bytes depending on your platform... – 0x26res May 14 '13 at 14:36
now i know why it wasn't working before even though the platforms on both machines are the same.. at the receiving end, winsock command recvfrom() was allocated only 4 bytes.. i did sizeof(Buffer) instead of sizeof(packet), where Buffer was char * and packet was a struct. Hence it was receiving only the first four bytes of data.. good to know about serialization.. really helped. – Vigo May 16 '13 at 08:19

score 1 · Answer 5 · answered May 14 '13 at 13:13

Ok I will take the example from the boost web site as I don't understand what you can not understand from it.
I added some comments and changes to it how you can transfer via network. The network code itself is not in here. For this you can take a look at boost::asio.

int main() {
    // create and open a character archive for output
    // we simply use std::strinstream here
    std::stringstream ofs;

    // create class instance
    const gps_position g(35, 59, 24.567f);

    // save data to archive
    {
        boost::archive::text_oarchive oa(ofs);
        // write class instance to archive
        oa << g;
        // archive and stream closed when destructors are called
    }

    // now we have const char* ofs.str().c_str()
    // transfer those bytes via network
    // read them on the other machine

    gps_position newg;
    {
        // create and open an archive for input
        std::stringstream ifs(the_string_we_read_from_the_network);
        boost::archive::text_iarchive ia(ifs);
        // read class state from archive
        ia >> newg;
        // archive and stream closed when destructors are called
    }
    return 0;
}

Serialization of struct

5 Answers5

Linked

Related