How to use multiprocess in another code and funcion?

Question

I am trying to multiprocess cell detection via the deepcell package. Because the deepcell detection works fine on small images. But for bigger images it does not work or takes a really really long time. So I'm trying to cut the images into small patches, and then use multiprocessing to feed them to the cell detection. I need to be able to run the pool_cell_detection() function in another code, and get its return value (allPoints). Whereas if I use it in a if name=='main' wrapper, then I could not get the return value. Can you suggest how I can do this?

Here is my code1

import numpy as np
import os
import matplotlib.pyplot as plt
import cv2 as cv
from multiprocessing import Pool
from deepcell.mesmer import Mesmer
import time

blevel_image = cv.imread("./images/blevel_eq_p.png",0)
app = Mesmer()


def deepcell_detection(image0, mpp):
    print(type(image0))
    cv.imwrite("./images/image1.png", image0)
    image = np.stack((image0,image0), axis=-1)
    image = np.expand_dims(image,0)
    labeled_image, coords = app.predict(image, image_mpp=mpp)
    print(len(coords))
    return coords

def pool_cell_detection(img_channel):
    blobs_log = []
    r,c = img_channel.shape[0:2]
    mpp = 2
    rstep=r//10
    cstep=c//10
    patches=[]
    for i in range(10):
        for j in range(10):
            img_patch = img_channel[i*rstep:(i+1)*rstep,j*cstep:(j+1)*cstep]
            patches.append([img_patch, mpp])
    with Pool(4) as p:
        print("pooling")
        allPoints=p.map(deepcell_detection, patches)
    return allPoints

def main():
    allPoints = pool_cell_detection(blevel_image)

if __name__ == '__main__':
   main()

I need in code 2 something like the following:

import code1

def func_something():
    #Many operations
    allPoints = pool_cell_detection(blevel_image)

But I'm not sure how to write code 2 to be able to get the allpoints

Answer 1

As the multiprocessing documentation says, the entry point of a multiprocessing program must be wrapped in if __name__ == '__main__': when using the spawn start method, which is the only available option on Windows, which you're on.

Change the final invocation from

blobs_log = pool_cell_detection(blevel_image)

to e.g.

def main():
    blobs_log = pool_cell_detection(blevel_image)

if __name__ == '__main__':
    main()

---

Following up on the comment threads, all in all, you might have, let's say, multiprocessing_cell_detect.py:

import multiprocessing

import cv2 as cv
import numpy as np
from deepcell.mesmer import Mesmer


def deepcell_detection(image0, mpp):
    cv.imwrite("./images/image1.png", image0)
    image = np.stack((image0, image0), axis=-1)
    image = np.expand_dims(image, 0)
    labeled_image, coords = Mesmer().predict(image, image_mpp=mpp)
    return coords


def pool_cell_detection(img_channel):
    r, c = img_channel.shape[0:2]
    mpp = 2
    rstep = r // 10
    cstep = c // 10
    patches = []
    for i in range(10):
        for j in range(10):
            img_patch = img_channel[i * rstep:(i + 1) * rstep, j * cstep:(j + 1) * cstep]
            patches.append([img_patch, mpp])
    with multiprocessing.Pool(4) as p:
        return p.map(deepcell_detection, patches)

and my_cell_program.py:

from multiprocessing_cell_detect import pool_cell_detection

def main():
    blevel_image = cv.imread("./images/blevel_eq_p.png", 0)
    allPoints = pool_cell_detection(blevel_image)
    print(allPoints)

if __name__ == '__main__':
    main()

and you run python my_cell_program.py.

So long as the main entry point of the program is guarded, things will work. The module you import a multiprocessing thing from will not need to be guarded (unless you also wish to use that module stand-alone).