Send std::vector over TCP, from boost::asio to QTcpSocket

Question

The server needs to send a std::vector<float> to a Qt application over a TCP socket. I am using Qt 5.7.

On the server side, using boost::asio:

std::vector<float> message_ = {1.2, 8.5};
asio::async_write(socket_, asio::buffer<float>(message_),
    [this, self](std::error_code ec, std::size_t)

This works and I manage to get it back on my client using boost::asio's read_some(). As both Qt and asio have their own event manager, I want to avoid using asio in my Qt app.

So on the client side I have (which does not work):

client.h:

#define FLOATSIZE 4
QTcpSocket *m_socket;
QDataStream m_in;
QString *m_string;
QByteArray m_buff;

client.cpp (constructor):

m_in.setDevice(m_socket);
m_in.setFloatingPointPrecision(QDataStream::SinglePrecision);
// m_in.setByteOrder(QDataStream::LittleEndian);

client.cpp (read function, which is connected via QObject::connect(m_socket, &QIODevice::readyRead, this, &mywidget::ask2read); ):

uint availbytes = m_socket->bytesAvailable(); // which is 8, so that seems good
while (availbytes >= FLOATSIZE)
{
    nbytes = m_in.readRawData(m_buff.data(), FLOATSIZE);

    bool conv_ok = false;
    const float f = m_buff.toFloat(&conv_ok);

    availbytes = m_socket->bytesAvailable();

    m_buff.clear();
}

The m_buff.toFloat() call returns 0.0 which is a fail according to the Qt doc. I have tried to change the float precision, little or big endian, but I can not manage to get my std::vector<float> back. Any hints?

Edit: everything runs on the same PC/compiler.

Edit: see my answer for a solution and sehe's for more detail on what is going on

@molbdnilo well a float is 4 bytes and a vector of two floats is 8 bytes, and see http://stackoverflow.com/a/22869206/7272199 . My understanding is that there is no serialization going on here, am I right ? — jmatthieu, Jan 18 '17 at 16:18
@molbdnilo yes it is (IOW asio does not _have_ a serialization format, that's your job) — sehe, Jan 18 '17 at 16:51

score 2 · Accepted Answer · edited May 23 '17 at 12:16

I managed to resolve the issue, by editing the Qt side (client), to read the socket:

uint availbytes = m_socket->bytesAvailable();
while (availbytes >= 4)
{
    char buffer[FLOATSIZE];
    nbytes = m_in.readRawData(buffer, FLOATSIZE);    
    float f = bytes2float(buffer);
    availbytes = m_socket->bytesAvailable();
}

I use those two conversion functions, bytes2float and bytes2int:

float bytes2float(char* buffer)
{
    union {
        float f;
        uchar b[4];
    } u;

    u.b[3] = buffer[3];
    u.b[2] = buffer[2];
    u.b[1] = buffer[1];
    u.b[0] = buffer[0];

    return u.f;
}

and:

int bytes2int(char* buffer)
{
    int a = int((unsigned char)(buffer[3]) << 24 |
        (unsigned char)(buffer[2]) << 16 |
        (unsigned char)(buffer[1]) << 8 |
        (unsigned char)(buffer[0]));
    return a;
}

I also found that function to display bytes, which is useful to see what is going on behind the scene (from https://stackoverflow.com/a/16063757/7272199):

template <typename T>
void print_bytes(const T& input, std::ostream& os = std::cout)
{
  const unsigned char* p = reinterpret_cast<const unsigned char*>(&input);
  os << std::hex << std::showbase;
  os << "[";
  for (unsigned int i=0; i<sizeof(T); ++i)
    os << static_cast<int>(*(p++)) << " ";
  os << "]" << std::endl;;
}

score 1 · Answer 2 · edited May 23 '17 at 12:01

Re. your answer: Which side is this on? Also, are your platforms not the same (OS/architecture?). I had assumed from the question that both processes run on the same PC and compiler etc.

For one thing, you can see that ASIO does not do anything related to endianness.

#include <boost/asio.hpp>
#include <iostream>
#include <iomanip>
namespace asio = boost::asio;

#include <iostream>

void print_bytes(unsigned char const* b, unsigned char const* e)
{
    std::cout << std::hex << std::setfill('0') << "[ ";
    while (b!=e)
        std::cout << std::setw(2) << static_cast<int>(*b++) << " ";
    std::cout << "]\n";
}

template <typename T> void print_bytes(const T& input) {
    using namespace std;
    print_bytes(reinterpret_cast<unsigned char const*>(std::addressof(*begin(input))), 
                reinterpret_cast<unsigned char const*>(std::addressof(*end(input))));
}

int main() {
    float const fs[] { 1.2, 8.5 };
    std::cout << "fs:     "; print_bytes(fs);

    {
        std::vector<float> gs(2);
        asio::buffer_copy(asio::buffer(gs), asio::buffer(fs));

        for (auto g : gs) std::cout << g << " "; std::cout << "\n";
        std::cout << "gs:     "; print_bytes(gs);
    }
    {
        std::vector<char> binary(2*sizeof(float));
        asio::buffer_copy(asio::buffer(binary), asio::buffer(fs));
        std::cout << "binary: "; print_bytes(binary);

        std::vector<float> gs(2);
        asio::buffer_copy(asio::buffer(gs), asio::buffer(binary));

        for (auto g : gs) std::cout << g << " "; std::cout << "\n";
        std::cout << "gs:     "; print_bytes(gs);
    }
}

Prints

fs:     [ 9a 99 99 3f 00 00 08 41 ]
1.2 8.5 
gs:     [ 9a 99 99 3f 00 00 08 41 ]
binary: [ 9a 99 99 3f 00 00 08 41 ]
1.2 8.5 
gs:     [ 9a 99 99 3f 00 00 08 41 ]

Theory

I suspect the Qt side ruins things. Since the naming of the function readRawData certainly implies a lack of endianness awareness, I'd guess your system's endianness wreaks havoc (https://stackoverflow.com/a/2945192/85371, also the comment).

Suggestion

In that case, consider using Boost Endian.

I edited my answer so it is clearer. Yes everything runs on the same PC/compiler. Thank you for your input it helps me understand better what is going on, I will have a look at boost endian to improve my code — jmatthieu, Jan 19 '17 at 09:25

score 0 · Answer 3 · answered Jan 18 '17 at 15:55

0

I think it's a bad idea to use high level send method server side (you try to send a c++ vector) and low level client side.

I'm quite sure there is an endianness problem somewhere.

Anyway try to do this client side:

char buffer[FLOATSIZE];
bytes = m_in.readRawData(buffer, FLOATSIZE);    

if (bytes != FLOATSIZE) 
     return ERROR;

const float f = (float)(ntohl(*((int32_t *)buffer)));

If boost::asio uses the network byte order for the floats (as it should), this will work.

answered Jan 18 '17 at 15:55

gabry

1,370
11
26

I get huge non-sense values for my floats using this, so asio might mess with the endianness – jmatthieu Jan 18 '17 at 16:49
The send method on the asio side is as lowlevel as it gets! It's bitwise serialization. It does *not* meddle with endianness. – sehe Jan 18 '17 at 16:52
@sehe would you have any clue of what is going on there ? – jmatthieu Jan 18 '17 at 16:57
I'd surely be writing answers then. Maybe if you can present a SSCCE client/server (I'll even install Qt for you) – sehe Jan 18 '17 at 16:58
@sehe I managed to work it out, writing my own conversion functions. thanks to both of you – jmatthieu Jan 18 '17 at 17:58

Send std::vector over TCP, from boost::asio to QTcpSocket

3 Answers3

Theory

Suggestion