2

I use ZMQ in python to distribute calculations between a master application and worker sub-processes, via a PUSH-PULL.

At times, the master might crash and the sub-processes remain hanging, listening to their respective ports.

I tried to use atexit to close the workers in the event that the master crashes, as suggested in this SO question. However atexit does not capture the case when I forcefully close the master.

Is there a way for the PULL-side worker sub-processes to notice that the PUSH-side master is closed via the zmq sock (as possibly hinted here)?

Practical Solution (edit)

A practical solution I implemented is to have the master PUSH a message to close all pending workers when it re-starts:

Before spawning its own helpers, the new instance of the master broadcasts an exit message to all sockets.

Upon receiving the exit command, the hanging sub-processes (launched by the previous instance of the master) do a sys.exit().

Community
  • 1
  • 1
Pythonic
  • 2,091
  • 3
  • 21
  • 34

1 Answers1

1

A: No, but workarounds exist

In case the sole PUSH-PULL:

Scaleable Formal Communication Pattern remains on the scene, then the answer has no other option to be but: no, cannot.

But ZeroMQ is a powerful mental shift into distributed processing concept

However, with some slight architecture shifts, the needed functionality comes in hand from other Formal Communication Patterns, deployed side-by-side with the initial PUSH-PULL solo.

enter image description here

The co-existent TransportPLANE(s) + SIG_PLANE(s) behavioural orchestration is limited only by one's imagination.

While not directly solving a code for your [dead-man button] signalling scenario, this answer illustrates the possible approaches in this direction by focusing on co-existing BEHAVIOUR(s) rather than on code.

Community
  • 1
  • 1
user3666197
  • 1
  • 6
  • 50
  • 92
  • Sounds good & works like a devil -- Yes, both **ZeroMQ** & **nanomsg** are devilish good frameworks and I love to work with 'em. Enjoy 'em too. – user3666197 Feb 03 '16 at 22:36