How to manage endianess of double from network

Question

I have a BIG problem with the answer to this question Swap bits in c++ for a double

Yet, this question is more or less what I search for: I receive a double from the network and I want to encoded it properly in my machine.

In the case I receive an int I perform this code using ntohl :

int * piData = reinterpret_cast<int*>((void*)pData);

//manage endianness of incomming network data 
unsigned long ulValue = ntohl(*piData);
int iValue = static_cast<int>(ulValue);

But in the case I receive an double, I don't know what to do.

The answer to the question suggest to do:

template <typename T>
void swap_endian(T& pX)
{
    char& raw = reinterpret_cast<char&>(pX);
    std::reverse(&raw, &raw + sizeof(T));
}

However , if I quote this site:

The ntohl() function converts the unsigned integer netlong from network byte order to host byte order. When the two byte orders are different, this means the endian-ness of the data will be changed. When the two byte orders are the same, the data will not be changed.

On the contrary @GManNickG's answer to the question always does the inversion with std::reverse .

Am I wrong considering that this answer is false ? ( in the extent of network management of endianess which the use of ntohl suggest though it was not precisely said in the title of the OP question).

In the end: Should I split my double into two parts of 4 bytes and apply the ntohl function on the two parts ? Are there more cannonical solutions ?

There's also this interesting question in C, host to network double?, but it limits to 32 bits values. And the answer says doubles should be converted to strings because of architecture differences... I'm also gonna work with audio samples, should I really consider converting all the samples to strings in my database ? ( the doubles come from a database that I query over the network)

It's not uncommon to have the way data is stored in a local database to be different than the way the data is serialized for network protocol. Is there a requirement that the data serialization be equivalent? — franji1, Feb 26 '13 at 01:02
Can't you combine the two solutions you linked to? `uint32_t foo = 1; if( htonl(foo) != foo ) { /* GMan's solution goes here */ } else { /* return unmodified argument */ }`. Also, the second answer is correct about floating point number representations being non-portable, even if endianness is not an issue. So should you convert to string, or use some specialized library for this? The answer depends on how portable you want this code to be. — Praetorian, Feb 26 '13 at 01:03
"the doubles come from a database that I query over the network" + libpq tag implies the standard postgresql access library, which should already be handling conversions for you... are you having troubles? Are you just assuming you need to do the conversion, and not seeing if the retrieved value is already usable? — Tony Delroy, Feb 26 '13 at 01:32
@TonyD the database is postgressql 9.1, and in the examples for libpq, one can find `ival = ntohl(*((uint32_t *) iptr));` when reading an integer value from a query cf. http://www.postgresql.org/docs/9.1/static/libpq-example.html so I suspect the convertion is not done if the postgresql documentation says to do that. And lastest time I work with postgres 8.3, on windows this time, I really needed to make the convertion for int, because I've had this bug ( not for double but for ints). — Stephane Rolland, Feb 26 '13 at 01:38
@StephaneRolland: yikes. Googling shows it's a common problem - what a library! You can see the encoding source code at http://doxygen.postgresql.org/pqformat_8c.html - see `pq_sendfloat8()` (there's a `pq_getmsgfloat8()` too but not obvious how to use it). Good luck hacking something up. — Tony Delroy, Feb 26 '13 at 02:09
@TonyD thx for the doxygen link :-) I'm gonna have a look at that. — Stephane Rolland, Feb 26 '13 at 09:05

John Källén · Accepted Answer · 2013-02-26T01:10:01.747

2

If your doubles are in IEEE 754 format that you should be relatively OK. Now you have to divide their 64 bits into two 32-bit halves and then transmit them in big-endian order (which is network order);

How about:

void send_double(double d) {
    long int i64 = *((reinterpret_cast<int *>)(&d)); /* Ugly, but works */
    int hiword = htonl(static_cast<int>(i64 >> 32));
    send(hiword);
    int loword = htonl(static_cast<int>(i64));
    send(loword);
}

double recv_double() {
    int hiword = ntohl(recv_int());
    int loword = ntohl(recv_int());
    long int i64 = (((static_cast<long int>) hiword) << 32) | loword;
    return *((reinterpret_cast<double *>(&i64));
}

edited Feb 26 '13 at 01:10

answered Feb 26 '13 at 01:03

John Källén

7,551
31
64

How can I know if the double are in IEEE-754 format ? is it the CPU, the linux version, or the database that constrains the encoding ? – Stephane Rolland Feb 26 '13 at 01:16
All of the above, but in the end, it is the database that decides what serialization format it will use. IEEE 754 has been around for many years now, but some database developers could rightly choose to be cautious and serialize doubles as strings so that they don't get bitten by subtle incompatabilies. – John Källén Feb 26 '13 at 01:23
The reinterpret_cast lines violate the struct aliasing rule. Also, `send_double` loses information if `sizeof(int) < sizeof(double)`, a common situation. – M.M Apr 27 '16 at 02:17
I would suggest using the `uint64_t` of `` (since C99 I think) or equivalenly `` (since C++11) as the 64-bit integer type for this kind of byte manipulations. The standard types like `int`, and `long int` may have diferent sizes in bytes depending on the compiler, as pointed out by Setepenre in one of the other answers. – jarzec Feb 26 '17 at 13:25
I had to use `static_cast(hiword)` in recv_double (can't put parentheses around the cast<>), send_double may need a similar change. – Lucas Walter Jun 01 '17 at 00:12

score 1 · Answer 2 · answered Feb 26 '13 at 01:03

1

Assuming you have a compile-time option to determine endianness:

#if BIG_ENDIAN
template <typename T>
void swap_endian(T& pX)
{
   // Don't need to do anything here... 
}
#else
template <typename T>
void swap_endian(T& pX)
{
    char& raw = reinterpret_cast<char&>(pX);
    std::reverse(&raw, &raw + sizeof(T));
}
#endif

Of course, the other option is to not send double across the network at all - considering that it's not guaranteed to be IEEE-754 compatible - there are machines out there using other floating point formats... Using for example a string would work much better...

answered Feb 26 '13 at 01:03

Mats Petersson

126,704
14
140
227

the database that will contain 8bytes float is a postgresql database, running on a debian or ubuntu 32 bits system. Is it the database, the linux version, or the Cpu that can say if the doubles are IEEE-754 compatible ? – Stephane Rolland Feb 26 '13 at 01:15
What I mean is that if you are receiving `double` data from another computer, what is to say that it's using the same type of `double` as yours? Yes, x86 processors do use IEEE-754 binary format. But some other machine, with a different model of floating point may not do that. If all the data is just going between your local machines that you have control over, you don't really need to worry about endianness in the first place - but if the data is from "something you don't know what it is", you also need to worry about "what does the bits mean" as well as the order of the bytes. – Mats Petersson Feb 26 '13 at 01:18

score 1 · Answer 3 · answered Aug 27 '15 at 22:02

I could not make John Källén code work on my machine. Moreover, it might be more useful to convert the double into bytes (8 bit, 1 char):

template<typename T>
string to_byte_string(const T& v)
{
    char* begin_ = reinterpret_cast<char*>(v);
    return string(begin_, begin_ + sizeof(T));
}

template<typename T>
T from_byte_string(std::string& s)
{
    assert(s.size() == sizeof(T) && "Wrong Type Cast");
    return *(reinterpret_cast<T*>(&s[0]));
}

This code will also works for structs which are using POD types.

If you really want the double as two ints

double d;
int* data = reinterpret_cast<int*>(&d);

int first = data[0];
int second = data[1];

Finally, long int will not always be a 64bit integer (I had to use long long int to make a 64bit int on my machine).

note: the code that uses these functions should be careful that it doesn't break if the string contains a null byte. — M.M, Apr 27 '16 at 02:21

score 0 · Answer 4 · answered Dec 08 '21 at 02:09

If you want to know system endianless

ONLY #if __cplusplus > 201703L

#include <bit>
#include <iostream>

using namespace std;

int main()
{
    if constexpr (endian::native == endian::big)
        cout << "big-endian";
    else if constexpr (endian::native == endian::little)
        cout << "little-endian";
    else
        cout << "mixed-endian";
}

For more info: https://en.cppreference.com/w/cpp/types/endian

How to manage endianess of double from network

4 Answers4

Linked