1

I have a function that sequentially searches for a particular feature in a list of image files, and terminates as soon as it finds a match. The function takes almost a second to run so I was hoping to employ multiprocessing to see if I can speed it up.

def match(a, b, feature) -> boolean:
  # time.sleep([0 - 10]) # simulates a long time-consuming function
  # return isMatch       # could be True or False depending on whether feature was found

p = Pool(processes=4)
list = p.map(match, [
  ['imgA', 'img1', feature],
  ['imgA', 'img2', feature],
  ['imgA', 'img3', feature],
  ['imgA', 'img4', feature],
  ['imgA', 'img5', feature],
  ...
  ...
  ['imgA', 'img100', feature],
])
found = next(x for x in list if x == True, False)

This works fine. BTW I am not actually hard-coding the parameters array. This is just for illustration purposes. However, I have a couple of questions regarding this approach:

  1. Right now, control doesn't return back to me until all 100 processes have completed execution, even if a match was found very early on. Is there a way I could signal from one of the child processes that it has succeeded so that the parent process can terminate all spawned processes and remove any pending processes in the queue?

  2. If that is possible, I would like to ensure that if that parent process was itself spawned, a child process experiencing success would not kill the parent's sibling process's child processes as well. In this example, I have kept a and feature constant. I would like to expand this solution so I can have both a and b as lists. In that case, for each a, I would spawn a child process to process a single a against all possible bs, which in turn should spawn grand-children that would process a single a against a single b. Every time I find a match between an a and a b, I would like to terminate all grand-children for the child process that was processing a single a against all possible bs.

If someone can guide me in the right direction, or even a library that gets this done for me, I would be really grateful. =)

Thanks a lot, Asim

AweSIM
  • 1,651
  • 4
  • 18
  • 37
  • Nevermind! I guess I didn't search SO well enough! =/ This question is a duplicate of https://stackoverflow.com/questions/54224989/killing-a-multiprocessing-process-when-condition-is-met?rq=1 – AweSIM Oct 23 '19 at 21:58

0 Answers0