How can I get a progress bar with a multiprocess (NOT a multiprocessing) pool?

Question

I am trying to get a tqdm-styled progress bar for a function that I'm parallelizing. When naively trying to set my map arguments as tqdm(vars), I get the following error:

from multiprocess import Pool
from time import sleep
import tqdm

def long_calculation(run):
    print(f'Starting run {run}')
    sleep(5)
    print(f'Ending run {run}')

vars_to_iterate = range(10)

with Pool(processes=6) as pool:
  pool.map(long_calculation, tqdm(vars_to_iterate))

Error that I get:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/var/folders/b4/n2km_tqj3cj7pvvx5kxn41nr0000gn/T/ipykernel_13567/113367457.py in <module>
     11 
     12 with Pool(processes=6) as pool:
---> 13   pool.map(long_calculation, tqdm(vars_to_iterate))

TypeError: 'module' object is not callable

All of the solutions to this issue, including the developer's response to the issue, seem to be for the library multiprocessing. However, I am looking for a solution that will work for the multiprocess library, because the former gives me this error when I try to use it in a Python Notebook.

Answer 1

This is not a multiprocess issue. It's a common pitfall with tqdm: you have to import the actual tqdm decorator from the tqdm module as

from tqdm import tqdm