Parallel Program error during pool creation in python

Question

this is the first time I am using the parallel program in python, my program is shown below:

np.random.RandomState(100)
arr = np.random.randint(0, 10, size=[200000, 5])#print(arr)
data = arr.tolist()
data[:5]

def howmany_within_range(row, minimum, maximum):    
count = 0
for n in row:
    if minimum <= n <= maximum:
        count = count + 1
return count

pool = mp.Pool(mp.cpu_count())
results = [pool.apply(howmany_within_range, args=(row, 4, 8)) for row in data]
pool.close()
print(results[:10])

the execution of this program give the following error:

Traceback (most recent call last): File "", line 1, in File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="mp_main") File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\109_personel\113_ARIMA_Tmin\Test_2.py", line 24, in pool = mp.Pool(mp.cpu_count())Traceback (most recent call last):

File "", line 1, in File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 119, in Pool File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 105, in spawn_main context=self.get_context()) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\pool.py", line 176, in init exitcode = _main(fd) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 114, in _main self._repopulate_pool() File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\pool.py", line 241, in _repopulate_pool prepare(preparation_data) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 225, in prepare w.start() File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py", line 112, in start _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path self._popen = self._Popen(self) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\context.py", line 322, in _Popen run_name="mp_main") return Popen(process_obj) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 263, in run_path

File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\popen_spawn_win32.py", line 33, in init prep_data = spawn.get_preparation_data(process_obj._name) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 143, in get_preparation_data pkg_name=pkg_name, script_name=fname) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 96, in _run_module_code _check_not_importing_main() File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main Traceback (most recent call last): File "", line 1, in mod_name, mod_spec, pkg_name, script_name) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.
exec(code, run_globals)

any one can help me to solve this error will be appreciated.

The solution is right in front of you. Add `if __name__ == '__main__':` like shown [here](https://stackoverflow.com/a/52693952/9059420). Also `pool.apply` won't help you here, [see](https://stackoverflow.com/q/53035293/9059420) and docs [here](https://docs.python.org/3.7/library/multiprocessing.html#multiprocessing.pool.Pool.apply_async) — Darkonaut, Feb 24 '19 at 00:16
Possible duplicate of [multiprocessing pool example does not work and freeze the kernel](https://stackoverflow.com/questions/52693216/multiprocessing-pool-example-does-not-work-and-freeze-the-kernel) — Darkonaut, Feb 24 '19 at 00:18
Any code that is not protected by `if __name__ == '__main__':`, gets executed again when a new process is started (the new process imports your code). — John Anderson, Feb 24 '19 at 00:44

Parallel Program error during pool creation in python

0 Answers0