Converting a structure containing floats into something more network friendly

Question

I am attempting to send a structure containing floating point data over a network, which comprises two different hardware architectures, written in C.

The client is running on x86_64 architecture, and the server is running on PPC (32bit) architecture.

It seems the only options I have for converting data to and from network friendly formats is:

htons/ntohs(2 byte) and htonl/ntohl(4 byte)

There appears to be no version which deals with floating point numbers (which makes sense, as differing architectures/endianness has different representations of floating point numbers).

So, I attempted splitting the floating point number into integer and exponent format in the following way:

void FtoME(float num, int32_t* mantissa, int32_t* exponent)
{
    int32_t sig = (int32_t)num;
    int32_t exp = 0;
    double frac = num-sig;
    double temp = num;
    while (frac > 0 && exp > -20)
    {
        temp = temp * 10; //left shift the whole number
        sig = (int32_t)temp; //get the integer part
        frac = temp - sig; //get the fractional part
        exp--;
    }
    printf("Scientific note: %dx10^%d\n",sig, exp);
    *mantissa = sig;
    *exponent = exp;
}

Now, whilst this does work in theory, in practice, I run into overflows a LOT, so clearly, this is not the correct way to handle this.

Are there other approaches I might try to be able to avoid overflows, and convert the float to a network friendly format (and importantly, back again), whilst not losing any data?

The only almost universal way to transmit data is plain UTF-8 encoded text. Binary data of any form will always have special or corner cases where it's just to much work to properly handle it in a portable way across all now living platforms and systems. — Some programmer dude, Jul 26 '19 at 12:18
Define the transmission protocol and stick to it. If your transmission format is "big endian 32-bit IEEE 754", then all parties will have to agree on this. You will likely need to use preprocessor defines if you plan on supporting weird architectures, but generally you can expect to have only endianess to worry about, i.e. [this answer to a very similar question](https://stackoverflow.com/a/10621440/69809). — vgru, Jul 26 '19 at 12:22
The issues are discussed [here](https://stackoverflow.com/questions/57168435/normalization-part-of-a-code-of-packing-a-float-ieee-754-into-uint64-t) and [here](https://stackoverflow.com/questions/4733147/portability-of-binary-serialization-of-double-float-type-in-c). Beware that you might the situation even worse if you invent another type of your own devising. By doing that you ensure you *never* have compatibility. — Weather Vane, Jul 26 '19 at 12:23
"When you feel the urge to design a complex binary file format, or a complex binary application protocol, it is generally wise to lie down until the feeling passes" -- Eric Steven Raymond in [The Art of Unix Programming, chapter 5](http://www.catb.org/~esr/writings/taoup/html/ch05s01.html) — pmg, Jul 26 '19 at 12:35
Possible duplicate of [Portable serialisation of IEEE754 floating-point values](https://stackoverflow.com/questions/10620601/portable-serialisation-of-ieee754-floating-point-values) — vgru, Jul 26 '19 at 12:56

score 4 · Answer 1 · answered Jul 26 '19 at 12:23

The IEEE 754 standard should be enough for most architectures - and for only those that are not compliant, you just need to worry about the conversion to IEEE 754 and back again. For the byteorder stuff, given a 32-bit float, you can use uint32_t tmp; memcpy(&tmp, &f, sizeof(f)); with htonl and ntohl.

Marco Bonelli · Answer 2 · 2019-07-29T23:37:52.047

There appears to be no version which deals with floating point numbers (which makes sense, as differing architectures/endianness has different representations of floating point numbers).

No it does not "make sense": every data type larger than one byte is going to be affected by the endianness of the system. This includes short and long integers.

The htonl and ntohl functions are exactly what you are looking for and can also be used to send floating point numbers, which are 32 bits (NOT doubles though, which are 64 bits... that's still doable, but a little more complicated).

The htonl function converts a 32 bit value from the host endianness to network endianness (which is big endian), while the ntohl function converts a 32 bit value from network endianness to host endianness. All you have to do is convert your float into a uint32_t appropriately when sending and when receiving, and you'll be good to go.

Server:

float f = 10e-9;
uint32_t tmp;

memcpy(&tmp, &f, 4);
uint32_t data = htonl(tmp);

send_to_client(data);

// Or if you are sending as raw bytes:
send_to_client((char*)&data, 4);

Client:

uint32_t data;

data = receive_from_server();

// Or if you are receiving as raw bytes:
char *bytes = receive_from_server(4);
memcpy(&data, bytes, 4);

float f = (float)ntohl(data);

As an example of dealing with a struct, assume you want to send this struct over a network:

struct data {
    char name[10];
    uint32_t something1;
    uint16_t something2;
    float value1;
    float value2;
};

Then you could do the following.

Server:

struct data x;
// Do something to initialize the fields of x.

char buffer[24]; // 10 + 4 + 2 + 4 + 4
uint32_t tmpl;
uint16_t tmps;

memcpy(buffer, x.name, 10);

tmpl = htonl(x.something1);
memcpy(buffer + 10, &tmpl, 4);

tmps = htons(x.something2);
memcpy(buffer + 14, &tmps, 2);

memcpy(&tmpl, &x.value1, 4)
tmpl = htonl(tmpl);
memcpy(buffer + 16, &tmpl, 4);

memcpy(&tmpl, &x.value, 4)
tmpl = htonl(tmpl);
memcpy(buffer + 20, &tmpl, 4);

send_to_client(buffer, 24);

Client:

char *buffer = receive_from_server(24);

struct data x;
uint32_t tmpl;
uint16_t tmps;

memcpy(x.name, buffer, 10);

memcpy(&tmpl, buffer + 10, 4);
x.something1 = ntohl(tmpl);

memcpy(&tmps, buffer + 14, 2);
x.something2 = ntohl(tmps);

memcpy(&tmpl, buffer + 16, 4);
tmpl = ntohl(tmpl);
memcpy(&x.value1, &tmpl, 4);

memcpy(&tmpl, buffer + 20, 4);
tmpl = ntohl(tmpl);
memcpy(&x.value2, &tmpl, 4);

This is great for sending a single value, but what if I have to send a couple of values, or a struct of values? — Ian Young, Jul 26 '19 at 12:39
@IanYoung then you'll have to do this for every value. You could just use a loop for example. Packing structures is also the same, just convert every single field to network endianness. I could expand my answer adding an example if you want. — Marco Bonelli, Jul 26 '19 at 12:40
Yes, an example of packing and sending structures would be appreciated, thank you. — Ian Young, Jul 26 '19 at 12:41
It is not safe at all to cast random locations to `uint32_t`, you should always use `memcpy`, see for example https://stackoverflow.com/questions/46790550/c-undefined-behavior-strict-aliasing-rule-or-incorrect-alignment — Antti Haapala -- Слава Україні, Jul 26 '19 at 13:20
@AnttiHaapala thank you again. Was reading about that in the other answer linked above. — Marco Bonelli, Jul 26 '19 at 13:27
Is `(unit32_t)f` correct? Won't this convert the float to an int truncating it? — Margaret Bloom, Jul 27 '19 at 13:23

score 0 · Answer 3 · answered Jul 28 '19 at 13:59

An example of a protocol that uses IEEE-754 (Network byte order or big endian) to transfer binary floating point is IPFIX. You can check how it does it.

Anyway, if you want something portable, you should select some encoding that's independent of architecture. I suggest you to use JSON as an example.

Converting a structure containing floats into something more network friendly

3 Answers3