I have recently started to use ZeroMQ and have observed that sometimes initialization of the ZeroMQ context can take as much as 40 seconds. Usually it's much faster than that.
Eg:
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
with zmq.Context.instance() as context:
# connect / bind to socket, do stuff
pass
This behaviour seems to be independent of whether or not I use a context manager, eg the following behaves the same as the above:
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
context = zmq.Context.instance()
# connect / bind to socket, do stuff
context.term()
This is running in Python 3.6.8 on Ubuntu 18.04.2
I haven't been able to establish any pattern to the slowdowns.
What are the possible causes of this behaviour?
Update
Per @user3666197 suggestion, I took some timings and managed to get the slowdown to manifest on the second run. Things returned to "normal" after that.
The slowdown kicked in on the second run of the test script:
= zmq.Context()] took + 704 [us] to execute
1.~ 704 us
2.~ 131 766 816 us
3.~ 145 us
4.~ 137 us
Watching memory on htop at the time did not show memory filling up at all but swap was already active.
Update 2
Using a pruned-down version of @user3666197's suggested script:
import sys
import zmq
clock = zmq.Stopwatch()
clock.start()
foo = zmq.Context()
bar = clock.stop()
print("{0:>2s}: {1:>9d}".format(sys.argv[1], bar))
foo.term()
And calling it in a loop, eg:
for x in $(seq 1 20);
do python3 test.zmq.py $x;
done
I can repeat the issue on the original server but have yet to hit it on another server, which suggests an environmental issue (or something to do with the specific distro version and/or python version as the "good" server is older).
This is sufficient for me for now but I will come back to it later and gather enough info to submit something to the maintainers.
