How to extract an argument when using an executor?

Question

What is the best way to extract arguments when calling function together. How can one function be run so that the other arguments get executed correctly?

How can each iterable be extracted when it is run by the executor?

Answer 1

As far as I know, there's no direct way to pass a dictionary and unpacking it using Executor.map.

You need to unpack it manually in single (also need to adjust the function to accept single paramter), or define wrapper function:

import concurrent.futures

def single(a, b):
    return (a, b)

def single_wrapper(d):  # <--- wrapper function to call single(**dict)
    return single(**d)

if __name__ == '__main__':
    iterable = [dict(a=1, b=2), dict(a=3, b=4)]
    with concurrent.futures.ProcessPoolExecutor() as executor:
        results = executor.map(single_wrapper, iterable)  # instead of `single`

    print(list(results))

---

Alternative: passing two iterables to map:

import concurrent.futures

def single(a, b):
    return (a, b)

if __name__ == '__main__':
    iterable = [dict(a=1,b=2), dict(a=3,b=4)]
    with concurrent.futures.ProcessPoolExecutor() as executor:
        iterable1 = [d['a'] for d in iterable]
        iterable2 = [d['b'] for d in iterable]
        results = executor.map(single, iterable1, iterable2)
        # iterable1[0], iterable2[0] will be passed to single as arguments.
        # iterable1[1], iterable2[1] will be passed to single as arguments.
        # ....

    print(list(results))