How to use parallelization in Python library function?

Question

I define a function FactorTest.py, which recieve a variable x and call function a():return x+1, b():return x+2, c():return x+3 and d():return x+4 in parallelized way.

Then I import FactorTest.py in another program as a library function. And when I call FactorTest and set x = 0, it get return [x+1,x+2,x+3,x+4] which is [1,2,3,4].

Is there any way can realize it? The point is in a parallelized way.

Answer 1

That's kind of not obvious thing. In case you are really going to do computing (which is CPU-bound operation) you can' benefit from concurrent approaches (asyncio and threading - because Python threads are limited GIL). It means, the only thing to approach it is using process, you can take a look at multiprocessing.Pool or ProcessPoolExecutor

The downside of this approach is cost of spawning a process. It has resource overhead, which means you may consider doing a few operations per one process, instead of running a dedicated process for each calculation.

import functools
import concurrent.futures

def action(x, y):
    return x + y

def factor(x, sum_range=4):
    result = []
    callback = functools.partial(action, x)
    with concurrent.futures.ProcessPoolExecutor() as executor:
        for num in executor.map(callback, range(1, sum_range+1)):
            result.append(num)
    return result


print(factor(2))
# > [3, 4, 5, 6]