0

I have used the following code as advised but still the pool is not terminated after first worker is finding a match. The desired result is to have 20 processes running function f and when any of them finds a match the pool should be terminated, what is happening instead is that I have to wait until all workers find a match.

from multiprocessing import Pool
import numpy as np


def f(z):..Output omitted


class Worker():
    def __init__(self, workers):
        self.pool = Pool(processes=workers)

    def callback(self, result):
        if result:
            self.pool.terminate()

    def do_job(self):

        self.pool.map_async(f,range(1,20), callback=self.callback)
        self.pool.close()
        self.pool.join()


if __name__=="__main__":
    w = Worker(20)
    w.do_job()
halfer
  • 19,824
  • 17
  • 99
  • 186
GREoIPsec
  • 23
  • 5
  • probably you could use multiprocessing.Event, but I don't know how exactly to use it with map – NobbyNobbs Oct 12 '18 at 15:00
  • 3
    Possible duplicate of [Python Multiprocess Pool. How to exit the script when one of the worker process determines no more work needs to be done?](https://stackoverflow.com/questions/33447055/python-multiprocess-pool-how-to-exit-the-script-when-one-of-the-worker-process) – noxdafox Oct 12 '18 at 15:31
  • Other possible duplicate: [How to get all pool.apply_async processes to stop once any one process has found a match in python](https://stackoverflow.com/questions/37691552/how-to-get-all-pool-apply-async-processes-to-stop-once-any-one-process-has-found/37700081#37700081) – noxdafox Oct 12 '18 at 15:32
  • Thanks for the feedback. I went over the links above but still I could not find a way to achieve my goal. I have tried to modify the code as above. Any idea? – GREoIPsec Oct 14 '18 at 17:26
  • What value does your `f` function return? Does it return the match when it finds it? Also, is it possible that your `f` function is returning a [falsy value](https://stackoverflow.com/questions/39983695/what-is-truthy-and-falsy-in-python-how-is-it-different-from-true-and-false) even if it finds a match? – bunji Oct 14 '18 at 19:35
  • The f function is running a while loop and it will not stop until it will hit True. – GREoIPsec Oct 14 '18 at 19:42
  • Ok but then does it return something or does it just stop? – bunji Oct 14 '18 at 19:53
  • Also it sounds like you want to use `apply_async` instead of `map_async`, see [this answer here](https://stackoverflow.com/a/19699363/5992438) for an explanation. `map_async` only calls the callback after every worker of the pool terminates – bunji Oct 14 '18 at 19:59
  • Hi, I have manage to solve the problem. I have called the w.callback(match) inside f so as soon as any process is finding a match in f the pool is terminated. Also I have replaced self.pool.map_async with self.pool.map and removed the self.pool.join(). Thank you! – GREoIPsec Oct 15 '18 at 07:41

0 Answers0