What is a good way to implement a barrier with ZeroMQ?

Question

I have a barrier implementation, using a PAIR/PAIR interconnection.

The server side will send a JSON-message to client and then wait for client sending a message. I will send 40000 messages in one test, and I repeat it for 10 times to calculate an average time used.

pair_Server.py :

import zmq
import random
import sys
import time

context = zmq.Context()
socket = context.socket(zmq.PAIR)
socket.bind("tcp://*:5556")

totalTime = 0
for testCount in range(10):
    isFirstSend = True
    startTime = 0
    for num in range(40000):
        socket.send_json({ 'num' : num })
        if isFirstSend:
            isFirstSend = False
            startTime = time.time()
        msg = socket.recv_json()
    totalTime += (time.time() - startTime)
    print("Finish test {}".format(testCount))

print(totalTime / 10)

pair_Client.py :

import zmq
import random
import sys
import time

context = zmq.Context()
socket = context.socket(zmq.PAIR)
socket.connect("tcp://localhost:5556")

while True:
    msg = socket.recv_json()
    socket.send_json(msg)

The output time spend is around 4.3 seconds. I then remove the barrier. Server now will only send JSON-message to client and do not need to wait for the reply. After 10 rounds. The average time spend is 0.3 seconds.

I understand that the speed will become faster but the difference is so big and I would like to ask that if I used a wrong implementation. I have tried PUSH/PULL sockets for a similar implementation and get a similar result.

user3666197 · Answer 1 · 2020-03-23T17:48:48.557

_{In case one has never worked with ZeroMQ,
one may here enjoy to first look at "ZeroMQ Principles in less than Five Seconds"
before diving into further details}

Q : "What is a good way to implement a barrier with ZeroMQ?"

The best way is to implement none such - performance-wise and latency-wise.

The as-is code does not implement a barrier, it enforces a local-side process to stop and stay in an unconstrained waiting-state for a remote, distributed-process to receive ( if ever ), process it ( if ever started and also hopefully finished ) and respond with at least some answer ( here, a delivery, if ever happens, at least a zero-length message is fine ) --before-- it will ever move a single step forwards.

That are pretty wild & principally unsafe assumptions, aren't they?

I would never accept a code ( not only for a production grade software ), that will knowingly accept itself to cause such a self-imprisonation into such an almost un-salvage-able mamooth-trap ( you yourself have lost control over your code-execution and the whole device it runs on ... yields a useless victim ... depending on unsure external coincidence of factors, that are completely out of your domain of control - possible, yet a bit un-convenient -- remember Apollo-11 landing on Moon ... a bad and ugly late moment to wait for Aha, ... isn't it? )

Your "barrier"-less code measures just the looping over throwing a batch of button-pushes ( without waiting, what each such button-push actually started to do - be it an automated-cannon firebutton, or a satellite remote-sensing camera push-button, having the satellite actually a few meters above the surface of the Ryugu asteroid, being many tens of light-minutes far away to ever hear your first button-"push"...

So the actual distributed-( behavioural )-protocol decides what may become a feasible solution and what will not.

Thank you for your answer. I read the thread you provided and I also read some other article `https://learning-0mq-with-pyzmq.readthedocs.io/en/latest/pyzmq/patterns/pushpull.html` for a basic understanding. I am sorry that I used a silly example but the main idea that I want to implement is that the server side should not go further if it did not receive a reply from client. What is a good way to implement it with ZMQ ? Thank you very much. — sflee, Mar 23 '20 at 23:45

What is a good way to implement a barrier with ZeroMQ?

1 Answers1