0

I have a python function that takes in an image path and outputs true or false depending whether the image is black or not. I want to process several images on the same machine and stop the process if even one of them is not black. I read a lot of multiprocessing in python, celery etc here, but I am not sure where to start.

alok
  • 49
  • 1
  • 8

2 Answers2

2

I would suggest looking at Pools to easily create processes on the fly. If you need to have some shared state, in this case a boolean indicating a non-black image has been found, look at Managers.

Update: Here is an example of what I mean.

import multiprocessing.Manager as Manager
import multiprocessing.Pool as Pool

m = Manager()
p = Pool(processes=5)

state_info = m.dict()
state_info['image_found'] = False

def processImage(img):

    # ... Process Image ...

    if imageIsBlack(img):
        state_info['image_found'] = True
        p.terminate()

 p.apply(processImage, imageList)

 if state_info['image_found']:
     print 'There was a black image!!'
 else:
     print 'No black images were found.'
owobeid
  • 173
  • 1
  • 9
  • I have a working code for spawning my processes and it is working fine but I am not able to exit out if functions for the process return False. – alok Feb 11 '13 at 20:09
  • If you are using Pools then you can just use terminate. I added an update to show you how. If you are sub-classing Process then be sure to check that 'image_found' is False before beginning the computation. – owobeid Feb 11 '13 at 22:07
  • Thanks for the code example but your example will raise an error becuase 'p' is not recognized as a variable in the scope of function 'processImage' and we cannot call p.terminate() from inside this function. Correct me if I am wrong. – alok Feb 11 '13 at 22:25
  • Also this function has no idea about state_info dict. – alok Feb 11 '13 at 22:31
  • Since p is defined outside the function it has a global scope. If you try to modify p, then you need to add `global p` at the beginning of processImage, but otherwise the above code should work. – owobeid Feb 11 '13 at 22:32
  • Ah I see . . . Since I am on windows, I have to use the if __name__=="__main__" to watchguard against nested imports. Thus my global scope is different from where I initialize manager and processes. – alok Feb 11 '13 at 22:37
2

Finally this works nicely for me. Copied it from an example here. For illustration purpose I have replaced my _isImgNonBlack function and the image sequence as a list of 0's and 1's, where 0 being a black image and 1 non-black image.

import multiprocessing

def isImgNonBlack(result_queue, imgSeq):
    for img in imgSeq:
        # If a non-black is found put a result
        if img==1:
            result_queue.put(1)

    # else put a zero as the result
    result_queue.put(0)

if __name__ == '__main__':
    processs = []
    result_queue = multiprocessing.Queue()
    nbProc = 20

    # making a fake list of images with 
    # 10,000 0's follwed by a single 1
    images = [0 for n in range(10000)]
    images.append(1)

    for n in range(nbProc): # start processes crawling for the result
        process = multiprocessing.Process(target=isImgNonBlack, args=[result_queue, images])
        process.start()
        processs.append(process)
        print 'Starting Process : %s' % process

    result = result_queue.get() # waits until any of the proccess have `.put()` a result

    for process in processs: # then kill them all off
        process.terminate()

    # finally print the result
    print "Seq have a non black img: %s" % result
Community
  • 1
  • 1
alok
  • 49
  • 1
  • 8