tqdm progress bar and multiprocessing

Question

I'm trying to add a progression bar to my program, however, solutions that seems to works for other (on other posts) do not work for me.

Python version 3.6.

import multiprocessing as mp
import tqdm

def f(dynamic, fix1, fix2):
    return dynamic + fix1 + fix2

N = 2

fix1 = 5
fix2= 10

dynamic = range(10)

p = mp.Pool(processes = N)

for _ in tqdm.tqdm(p.starmap(f, [(d, fix1, fix2) for d in dynamic]), total = len(dynamic)):
    pass

p.close()
p.join()

Any idea why the multiprocessing works (the computation is done), but there is no progress bar?

NB: The example above is dummy, my function are different.

Other question: how can I interrupt properly a multiprocessing program? The ctrl+C that I usually do in signle thread seems to pose some issues.

Answer 1

Unfortunately, tqdm is not working with starmap. You can use the following:

def f(args):
  arg1, arg2 = args
  ... do something with arg1, arg2 ...


for _ in tqdm.tqdm(pool.imap_unordered(f, zip(list_of_args, list_of_args2)), total=total):
    pass