ZeroMQ Poller vs Tornados EventLoop

Question

Design wise and performance wise which approach is recommended for handling multiple Zeromq sockets and why ?

Is it true that Tornado's IOLoop used by ZeroMQ hogs less CPU than the Poller used in a while loop for handling multiple sockets ?

score 4 · Accepted Answer · answered Jun 25 '12 at 15:51

4

It would be nice if you add your own observation / analysis to your question.

I don't think there is any difference in performance but there is a difference in design.

In case of poller

You register your sockets to be polled and then you use if blocks to identify and work with each socket.

while should_continue:
        socks = dict(poller.poll())
        if socket_pull in socks and socks[socket_pull] == zmq.POLLIN:
            Work_on_socket_pull ....

        if socket_sub in socks and socks[socket_sub] == zmq.POLLIN:
             Work_on_socket_sub ....

In case of eventloop But using event loop is quite elegant when you are handling multiple sockets since you register callbacks.

stream_pull = zmqstream.ZMQStream(socket_pull)
stream_pull.on_recv(getcommand)

stream_sub = zmqstream.ZMQStream(socket_sub)
stream_sub.on_recv(process_message)

As you can notice from the second example, the if blocks are removed. You write your socket messaging operation else where and you register your callback methods when socket is ready. In this case on_recv()

I hope that clarifies your question.

answered Jun 25 '12 at 15:51

pyfunc

65,343
15
148
136

Thanks for your reply. It is obvious from your answer that it might make sense to use the eventloop since the system of callbacks makes the code more readable. I am not sure which is more efficient - Since the eventloop uses the tornado IOLoop does it also make things faster ? – crashekar Jun 25 '12 at 16:46
@crashekar : No I don't think it adds to performance but is more elegant as I suggested in my answer, when you are writing more complex pieces of information. Your looping is handled by Tornado IO loop and it does the selection and calling your callbacks. In a more complex project, this is elegant because you also register other events with IO loop and call them too through the same design interface – pyfunc Jun 25 '12 at 17:10
Let me dig more into the performance part of my question - is it true that the while loop - poller construct hogs more CPU and IOloop is less CPU intensive ? – crashekar Jun 26 '12 at 05:45
@crashekar: Polling is supposed to be non-busy wait. There should not be any cpu usage while waiting on sockets. Have you faced any perceptible performance issues? – pyfunc Jun 26 '12 at 06:27

score 1 · Answer 2 · answered Jun 26 '12 at 12:00

1

The Tornado IO Loop, as reimplemented by PyZMQ, uses the poller behind the scenes anyway, so it is unlikely that one would use more CPU than the other.

See https://github.com/zeromq/pyzmq/blob/master/zmq/eventloop/ioloop.py for details.

answered Jun 26 '12 at 12:00

R Hyde

10,301
1
32
28

ZeroMQ Poller vs Tornados EventLoop

2 Answers2