Sorting memory fixing Byte order Endian

Question

I am struggeling with a problem on an embedded system. I have to program in C which is quite the task for me as i usualy work in object based programming languages. In this system two processors share memory and they handle data differntly, i am pretty sure the issue has to do with endian encoding, but after 3 hours of failed attempts to make a single function to fix this i am seeking for help. I struggle immensly with casting and and pointer handeling.

The situation: I store two (or more) floats (or other types) in a buffer lets say for this example its {1.0f , 1.0f}. This ends up with the byte representation of {3F,80,00,00,3F,80,00,00} if the other processor reinterprets the data it gets negative or rediculously high exponential numbers depending on the input.

So what am i trying to achive? A function that fixes this order where the endian encoding is reversed.

void fixEncoding(void *StoredData, uint8_t variableSizeInBytes, uint8_t buffer size)

The parameters would be a pointer to my stored data {3F,80,00,00,3F,80,00,00} the size of the variables encoded within (4 for a float for example or sizeof(float)) and the stored datas size

input {3F,80,00,00,3F,80,00,00} , 4, 8
output{00,00,80,3F,00,00,80,3F}

input {3F,80,00,00,3F,80,00,00} , 2, 8
output{80,3f,00,00,80,3f,00,00}

You're making your life harder by passing the variable size as a parameter. Start by writing a `void fixFloat(void *StoredData, uint8_t count)` function, where `count` is the number of floats in the input buffer. Once you get that working, you can decide whether you actually need other sizes. — user3386109, Jan 13 '22 at 09:11
Why do you need custom function? Can't you use system specific functions like `ntohl` or `htonl` for changing the endianness? — kiner_shah, Jan 13 '22 at 09:46
If two processors share the same memory, you have to specify a "network endianess" for everything stored in that memory. Then each CPU has to fall in line and if necessary re-order data before writing to that memory. — Lundin, Jan 13 '22 at 10:17
@kiner_shah Because those functions are unlikely available on an embedded system and it's hardly rocket science to implement them either. It's just a for loop. I never quite understood why *nix programmers need a library call for such trivial stuff... and of course always name the function something cryptic like `hrpdrp` instead of `little_to_big`. — Lundin, Jan 13 '22 at 10:19
@Lundin, there are better ways than for loops. For example, https://stackoverflow.com/a/51699282/4688321 — kiner_shah, Jan 13 '22 at 10:27
@kiner_shah Sure, shifts could be slightly more efficient if you know the size of the variable. But shifting also assumes that you know which endianess you are using and which one you want to end up with. `data[n] = (old >> 24) & 0xFF;` n=0 if big endian, n=3 or so if little endian. The bit shift itself is endianess-independent, but the data indices aren't. — Lundin, Jan 13 '22 at 10:31
@Jabberwocky better ask what i didnt try... i wasnt really struggeing with the loop itself but was looking to get a generic solution to this problem and something the compiler would let me do, as the idea of a void pointer confused my object orientated brain. Its only ever very small amounts of data that are beeing shared by the systems so the answers below more than satisfy my need. I agree that i could have included my attempts and i had them pasted in the question but it didnt really contribute to this beeing a good or clear question so i did not include it. — Denis Schaf, Jan 13 '22 at 14:02
And admitingly the version were you always just swap the last and first byte without storing the whole thing is something i didnt come up with in my attempts so i am pretty happy nobody fixed my loop, but rather showed me how to do it better — Denis Schaf, Jan 13 '22 at 14:05
@user3386109 i had that, but comming from C# i dint feel like haveing many functions for all the different data types but rather looked for something generic that i can reuse — Denis Schaf, Jan 13 '22 at 14:13

score 2 · Accepted Answer · answered Jan 13 '22 at 09:35

Swapping bytes in an element is a common idiom: you go up to the half of the bytes and exchange with the symetric byte. You just have to iterate your buffer as bytes, element by element.

Code could be:

void fixEncoding(void* StoredData, uint8_t variableSizeInBytes, uint8_t buffer_size) {
    char* data = StoredData;    // must be char to be able to use pointer arithmetics
    while (buffer_size > 0) {
        int i;
        // swap the element at data
        for (i = 0; i < variableSizeInBytes / 2; i++) {
            int j = variableSizeInBytes - i - 1;
            char t = data[i];
            data[i] = data[j];
            data[j] = t;
        }
        // skip to next element
        data += variableSizeInBytes;
        buffer_size -= variableSizeInBytes;
    }
}

I am amased this works with only one cast. My c code seems to be a convoluted casting mess. Thanks for that! — Denis Schaf, Jan 13 '22 at 14:22

score 1 · Answer 2 · answered Jan 13 '22 at 09:50

First you need to decide on what byte order you want to be the common byte order and then determine if the bytes in the buffer should be reversed. One of the processors already store floats in this common byte order so for that processor, you never need to reverse the bytes in the buffer.

I suggest using network byte order just because there are lots of examples of converting to/from that format.

I'm here using htonl (host to network byte order for uint32_t) to determine if the processor is already using this order, but there are more effective ways. See C Macro definition to determine big endian or little endian machine?.

Example:

#include <arpa/inet.h> // htonl

void swap_uint8_t(char *a, char *b) { // swap two bytes
    uint8_t tmp = *a;
    *a = *b;
    *b = tmp;
}

void reverse(void *data, size_t len) { // reverse one `float` (or whatever you have)
    uint8_t *s = data;
    uint8_t *e = s + len;
    for(;s < e--; ++s) {
        swap_uint8_t(s, e);
    }
}

void fixEncoding(void *StoredData, uint8_t variableSizeInBytes, uint8_t buffer_size) {
    if(htonl(1) != 1) {  // determine if this machine is not using network byte order
        // not network byte order, reverse each `float` (or whatever you have)
        uint8_t *data = StoredData;
        for(uint8_t pos = 0; pos < buffer_size; pos += variableSizeInBytes) {
            reverse(data + pos, variableSizeInBytes);
        }
    }
}

Demo

This is not 100% safe since some uncommon processors uses mixed endianess. Check that yours don't.

There's no obvious reason why one would ever need to do this in-place, instead of just creating a new variable, since it will never be larger than 8 bytes. `uint8_t* ptr = (uint8_t*)&old; for(size_t i=0; i — Lundin, Jan 13 '22 at 10:27
this one works as well but Serges answer came first. Still, please have my upvote good sir! — Denis Schaf, Jan 13 '22 at 10:31
@Lundin True. I was hoping that it would be easy to read and understand this way. It's not at all optimized for speed. — Ted Lyngmo, Jan 13 '22 at 10:34
Switching endianess is almost always something done "in series" with some data communication protocol though, so it's likely to be somewhat performance-critical. If done on a low level, as in a driver grabbing code from some bus interface, the endianess can be fixed at the same time as you copy the data from the buffer in the bus hardware peripheral into RAM. — Lundin, Jan 13 '22 at 10:37

Sorting memory fixing Byte order Endian

2 Answers2