Store C structs for multiple platform use - would this approach work?

Question

Compiler: GNU GCC

Application type: console application

Language: C

Platforms: Win7 and Linux Mint

I wrote a program that I want to run under Win7 and Linux. The program writes C structs to a file and I want to be able to create the file under Win7 and read it back in Linux and vice versa.

By now, I have learned that writing complete structs with fwrite() will give almost 100% assurance that it won't be read back correctly by the other platform. This due to padding and maybe other causes.

I defined all structs myself and they (now, after my previous question on this forum) all have members of type int32_t, int64_t and char. I am thinking about writing a WriteStructname() function for each struct that will write the individual members as int32_t, int64_t and char to the outputfile. Likewise, a ReadStructname() function to read the individual struct members from the file and copy them to an empty struct again.

Would this approach work? I prefer to have maximum control over my sourcecode, so I'm not looking for libraries or other dependencies to achieve this unless I really have to.

Thanks for reading

Answer 1

Element-wise writing of data to a file is your best approach, since structs will differ due to alignment and packing differences between compilers.

However, even with the approach you're planning on using, there are still potential pitfalls, such as different endianness between systems, or different encoding schemes (ie: two's complement versus one's complement encoding of signed numbers).

If you're going to do this, you should consider something like a JSON parser to encode and decode your data so you don't corrupt it due to the issues mentioned above.

Good luck!

Answer 2

If you use GCC or any other compiler that supports "packed" structs, as long you avoid yourself from using anything but [u]intX_t types in the struct, and execute endianness fix in any field where type is bigger than 8 bits, you are platform safe :)

This is an example code where you get portability between platforms, do not forget to manually edit the endianness UIP_BYTE_ORDER.

#include <stdint.h>
#include <stdio.h>

/* These macro are set manually, you should use some automated detection methodology */
#define UIP_BIG_ENDIAN 1
#define UIP_LITTLE_ENDIAN 2
#define UIP_BYTE_ORDER UIP_LITTLE_ENDIAN

/* Borrowed from uIP */
#ifndef UIP_HTONS
#   if UIP_BYTE_ORDER == UIP_BIG_ENDIAN
#      define UIP_HTONS(n) (n)
#      define UIP_HTONL(n) (n)
#      define UIP_HTONLL(n) (n)
#   else /* UIP_BYTE_ORDER == UIP_BIG_ENDIAN */
#      define UIP_HTONS(n) (uint16_t)((((uint16_t) (n)) << 8) | (((uint16_t) (n)) >> 8))
#      define UIP_HTONL(n) (((uint32_t)UIP_HTONS(n) << 16) | UIP_HTONS((uint32_t)(n) >> 16))
#      define UIP_HTONLL(n) (((uint64_t)UIP_HTONL(n) << 32) | UIP_HTONL((uint64_t)(n) >> 32))
#   endif /* UIP_BYTE_ORDER == UIP_BIG_ENDIAN */
#else
#error "UIP_HTONS already defined!"
#endif /* UIP_HTONS */


struct __attribute__((__packed__)) s_test
{
    uint32_t a;
    uint8_t b;
    uint64_t c;
    uint16_t d;
    int8_t string[13];
};

struct s_test my_data =
{
    .a = 0xABCDEF09,
    .b = 0xFF,
    .c = 0xDEADBEEFDEADBEEF,
    .d = 0x9876,
    .string = "bla bla bla"
};

void save()
{
    FILE * f;
    f = fopen("test.bin", "w+");

    /* Fix endianness */
    my_data.a = UIP_HTONL(my_data.a);
    my_data.c = UIP_HTONLL(my_data.c);
    my_data.d = UIP_HTONS(my_data.d);

    fwrite(&my_data, sizeof(my_data), 1, f);
    fclose(f);
}

void read()
{
    FILE * f;
    f = fopen("test.bin", "r");
    fread(&my_data, sizeof(my_data), 1, f);
    fclose(f);

    /* Fix endianness */
    my_data.a = UIP_HTONL(my_data.a);
    my_data.c = UIP_HTONLL(my_data.c);
    my_data.d = UIP_HTONS(my_data.d);
}

int main(int argc, char ** argv)
{
    save();
    return 0;
}

Thats the saved file dump:

fanl@fanl-ultrabook:~/workspace-tmp/test3$ hexdump -v -C test.bin 
00000000  ab cd ef 09 ff de ad be  ef de ad be ef 98 76 62  |..............vb|
00000010  6c 61 20 62 6c 61 20 62  6c 61 00 00              |la bla bla..|
0000001c

Answer 3

This is a good approach. If all fields are integer types of a specific size such as int32_t, int64_t, or char, and you read/write the appropriate number of them to/from arrays, you should be fine.

The one thing you need to watch out for is endianness. Any integer type should be written in a known byte order and read back in the proper byte order for the system in question. The simplest way to do this is with the ntohs and htons functions for 16-bit ints and the ntohl and htonl functions for 32-bit ints. There's no corresponding standard functions for 64-bit ints, but that shouldn't be to difficult to write.

Here's a sample of how you could write these functions for 64 bit:

uint64_t htonll(uint64_t val)
{
    uint8_t v[8];
    uint64_t *result = (uint64_t *)v;
    int i;

    for (i=0; i<8; i++) {
        v[i] = (uint8_t)(val >> ((7-i) * 8));
    }

    return *result;
}

uint64_t ntohll(uint64_t val)
{
    uint8_t *v = (uint8_t *)&val;
    uint64_t result = 0;
    int i;

    for (i=0; i<8; i++) {
        result |= (uint64_t)v[i] << ((7-i) * 8);
    }

    return result;
}