How to use concurrent.futures in Python

Question

Im struggling to get multithreading working in Python. I have i function which i want to execute on 5 threads based on a parameter. I also needs 2 parameters that are the same for every thread. This is what i have:

from concurrent.futures import ThreadPoolExecutor

def do_something_parallel(sameValue1, sameValue2, differentValue):
    print(str(sameValue1))        #same everytime
    print(str(sameValue2))        #same everytime
    print(str(differentValue))    #different

main(): 

    differentValues = ["1000ms", "100ms", "10ms", "20ms", "50ms"]

    with ThreadPoolExecutor(max_workers=5) as executor:
        futures = [executor.submit(do_something_parallel, sameValue1, sameValue2, differentValue) for differentValue in differentValues]

But i don't know what to do next

score 6 · Accepted Answer · edited Nov 26 '18 at 15:22

6

If you don't care about the order, you can now do:

from concurrent.futures import as_completed

# The rest of your code here

for f in as_completed(futures):
    # Do what you want with f.result(), for example:
    print(f.result())

Otherwise, if you care about order, it might make sense to use ThreadPoolExecutor.map with functools.partial to fill in the arguments that are always the same:

from functools import partial

# The rest of your code...

with ThreadPoolExecutor(max_workers=5) as executor:
    results = executor.map(
        partial(do_something_parallel, sameValue1, sameValue2),
        differentValues
    ))

edited Nov 26 '18 at 15:22

rAntonioH

129
2
12

answered Mar 19 '18 at 08:47

samfrances

3,405
3
25
40

How can this be done using keyword arguments? Trying `executor.map(partial(fun, param1=arg1, param3=arg3), param2=arg2)` throws an error, and removing `param2=` leads to an error as the iterable argument is passed to the first parameter. – Buzz B Jul 18 '23 at 15:10

How to use concurrent.futures in Python

1 Answers1

Linked

Related