1

The classic ZeroMQ PUB pattern, is something like :

  1. format your complete message
  2. send your message
  3. ( managed by ZMQ ) if there is a subscriber to the topic, then send it, else trash it ?

What I've noticed in one of my applications, is that the formatting of some of the messages is very heavy and takes a lot of time. When I don't have a subscriber for the topic, I do all this work for nothing.

I was wondering if there was a way to check whether a topic was subscribed before formatting the rest of the message.

I understand there'd be a TOCTOU problem :
1. check the topic is subscribed ( it's not )
2. ( ZMQ receives a subscription for the topic )
3. data is not sent...

or
1. check the topic is subscribed ( it is )
2. start formatting message
3. ( ZMQ receives a un-subscription for the topic )
4. send to socket, data is not sent ( wasted time )

... and I'm OK with both.

I've tried with multi-part messages ( sending first the "header/topic" without formatting the rest of the message ) but :
- it doesn't seem to do what I'm meaning here
- my subscribers also have to handle the multi-part messages ( can do a simple zmq_recv() ), which is a bit annoying

Any idea ? I think I see where to patch in xpub.cpp , adding a method that would copy/paste part of xpub::xsend() ( https://github.com/zeromq/libzmq/blob/656205b5f9159677d325cff5e6e26c97f95d8cd7/src/xpub.cpp#L289 ) but I'm not even sure that's something the ZMQ community would be interested into.

user3666197
  • 1
  • 6
  • 50
  • 92
Touisteur EmporteUneVache
  • 111
  • 1
  • 8

1 Answers1

1

In case one has never worked with ZeroMQ,
one may here enjoy to first look at "ZeroMQ Principles in less than Five Seconds"
before diving into further details

Q : "Can we check subscribers before sending a message?"

Yes, we can.

If indeed in such a need, beware the XPUB Archetype collects incoming subscription-management messages ( if they arrive ) usable for doing something like this.

That does not mean one can stand blind and rely on this. Unless in a fully-restricted environment, where rigid version-control and enforcement policies are strong & in-place, there always may be a client, that does not use the more recent, changed, version, that performs the topic-filtering on (X)PUB-side. Given such chance, the SUB-side topic-filtering ought be fully simulated, if it delivers all the subscription-management records onto the (X)PUB-side, as the newer versions expect, before starting to blind-sightedly "believe" into such a test-before-send policy is being adopted.

Damned version management :o)

You may also know, that the topic-filtering ( since ever and hopefully will remain so ) does not require any formatting the less a multi-part messaging overheads. It works as a plain bit-field matching, the performance of which was tuned-up, so who would ever want to waste any single [ns] of some add-on overhead costs in this domain?

Welcome to the Art of Zen-of-Zero

user3666197
  • 1
  • 6
  • 50
  • 92
  • Hi, thanks for this. Do you mean I'd have to reimplement the topic-reception-and-filtering feature of the pub socket ? – Touisteur EmporteUneVache Mar 11 '20 at 15:55
  • Did I say anything near this?No.The text tried,with all limitations of my knowledge&my writing style,that given the documented published API-properties, the user-level code (apps) shall follow the known properties,how the `Context()`-instances work.The topic-filtering was always and in detail described in either version of each of the released API-specifications.The co-existence of different versions is a fact, one can hardly avoid.Any&all differences of doing a local-side topic-filtering (in some) or network-remote traffic & deferred filtering (in other) matter.Zero-space for speculations :o) – user3666197 Mar 11 '20 at 16:36