6

The example way to send messages using Cap'n Proto needs a file descriptor to write to:

     ::capnp::writeMessageToFd(fd, message);

But in ZMQ the message needs to be passed to a ZMQ function:

    zmq_send(requester, "Hello", 5, 0);

http://zguide.zeromq.org/page:all

How can this incompatibility be resolved?

BAR
  • 15,909
  • 27
  • 97
  • 185

1 Answers1

11

Two possibilities:

  1. Use capnp::messageToFlatArray() to get the message as a single flat array. Note that this requires making a copy of the message contents.
  2. Send the message as a ZeroMQ multi-part message, with the parts being the message's segments. capnp::MessageBuilder::getSegmentsForOutput() returns an array of arrays pointing into the raw message segments. capnp::SegmentArrayMessageReader takes such an array of arrays as input. If you can send the array of arrays as a multipart message, then you can skip using capnp/serialize.h at all, since its only purpose is to combine the segments into a single message with a segment table. In your case, ZeroMQ would be in charge of remembering where each segment starts and ends.

I recommend #2, but it is somewhat more complicated.

Kenton Varda
  • 41,353
  • 8
  • 121
  • 105
  • Thanks, I will have to try it out. I was considering a situation where I would want to send N messages at the same time over the socket (possible perf boost) but there is no simple way to bundle the Cap'n Proto messages to call `writeMessageToFd` just once and write all N messages. It looks like this could be accomplished with your answer but with some relative complexity added. Any particular reason why you programmed it this way? I would think this is to allow zero-copy messaging, which I think is great, but it would help to have built-in helpers for the alternative approach. – BAR Aug 17 '15 at 03:32
  • I don't think writing multiple messages "at once" will improve performance much. If you're worried about sending too many small packets, use TCP_CORK. If you think syscall overhead is a problem, you can write to a `kj::BufferedOutputStream` which will do an extra copy in-memory but fewer syscalls. But it would surprise me if this actually helps. Alternately, depending on the structure of your app, perhaps it makes sense to create a wrapper struct that contains a list of your message type, so that you can bundle multiple messages into one. This will save on memory allocations. – Kenton Varda Aug 17 '15 at 16:46