0

In the Python Queue documentation, there is the following example:

def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

q = Queue()
for i in range(num_worker_threads):
     t = Thread(target=worker)
     t.daemon = True
     t.start()

for item in source():
    q.put(item)

q.join()       # block until all tasks are done

I'm just wondering if this is the right approach, since it seems that the worker threads will never terminate. I think they'll become zombies in the end. Is it an issue that Python doc overlooked? Thanks.

J Freebird
  • 3,664
  • 7
  • 46
  • 81

1 Answers1

1

It depends on what exactly your threads are doing and if they should be allowed to be terminated in the middle of doing one of their jobs.

Since you're using daemon = True the child threads should be killed when your parent ends. (More info in the docs here)

On the other hand, if you want to do a bit cleaner of a cleanup, you should probably not use while True and instead use while not parent_said_stop, removing the daemon = True. Then depending on how you stop the parent (whether it be through some key click, an interrupt, or whatever else not), you can catch the fact that the parent is stopping and have the message be sent to the children. (Ways of doing things when the process ends can be seen here)

Community
  • 1
  • 1
Jenny Shoars
  • 994
  • 3
  • 16
  • 40
  • The above example is using multithreading, but you're talking about multiprocessing here. A daemon thread will not exit even if its parent exits, which is different from a daemon process. – J Freebird Jan 23 '16 at 23:01
  • @JFreebird, I think the same thing applies to threads from what I can tell from the [docs](https://docs.python.org/3/library/threading.html#thread-objects). If only daemon threads are left, they are all abruptly terminated. And so both the approaches I described should still be valid... At least, I think. – Jenny Shoars Jan 23 '16 at 23:11