Python multiprocessing gets stuck

Question

I am pretty new to Python and totally new to multiprocessing, I found a few tutorials online to help me understand the multiprocessing package.

My code has a set of differential equations using RungeKutta4 method and I need to run tons of calculations with different starting conditions.

The code itself works but without multiprocessing it takes long to finish and I thought it might be ideal to use parallelization because the calculations are independent....

I am using Anaconda as IDE btw..

So I used

import multiprocessing as mp
iterations = np.arange(start,end,step)
pool = mp.Pool(mp.cpu_count())                           # Step 1: Init multiprocessing.Pool()
results = pool.map(rungeKutta4, [k for k in iterations]) # Step 2: apply pool map              
pool.close()                                             # Step 3: close

When I run it in Anaconda I dont get an error, it starts calculating but it never stops.... Where did I go wrong?

Thanks in advance for help...

Best

EDIT: I added the whole code here...

# Python program to implement Runge Kutta method 
# Markus Schmid
# 2020 Appalachian State University
# jupyter nbconvert --to python FILENAME.ipynb

# y" + 2*beta*y' + w0*sin(y) = A + B*cos(w*t)
# Import used libraries
import numpy as np
import math
import matplotlib.pyplot as plt
import time
from matplotlib import rc
rc('font',**{'family':'sans-serif','sans-serif':['Helvetica']})
rc('text', usetex=True)
import multiprocessing as mp
print("Number of processors: ", mp.cpu_count())
# list for time (x axis) and result (y axis)
result = []
w0 = 10                  # undamped angular frequency of the oscillator
beta = 1
B = 1
A = 0
B = 10
w = 4
theta_init = 0
theta_end = 20
theta_step = 0.1

# initial conditions
t0 = 0
y0 = 0
z0 = 0
t_final = 20                   # final time
h = 0.01                    # fixed step size
n = (int)((t_final - t0)/h)   # datapoints per RK4 iteration

# define functions
def funcf(t, y, z): 
    return (z)
def funcg(t, y, z): 
    return (A+B*math.cos(w*t) - 2*beta*z - w0*math.sin(y))

# Finds value of y for a given x using step size h 
# and initial value y0 at x0. 
def rungeKutta4(y): 
    # Count number of iterations using step size or 
    # step height h 
    t = t0
    z = z0
    n = (int)((t_final - t)/h)  
    for i in range(1, n + 1): 
        # Apply Runge Kutta to find next value of y
        k1 = h * funcf(t, y, z) 
        l1 = h * funcg(t, y, z)  

        k2 = h * funcf(t + 0.5 * h, y + 0.5 * k1, z + 0.5 * l1) 
        l2 = h * funcg(t + 0.5 * h, y + 0.5 * k1, z + 0.5 * l1)  

        k3 = h * funcf(t + 0.5 * h, y + 0.5 * k2, z + 0.5 * l2) 
        l3 = h * funcg(t + 0.5 * h, y + 0.5 * k2, z + 0.5 * l2) 

        k4 = h * funcf(t + h, y + k3, z + l3) 
        l4 = h * funcg(t + h, y + k3, z + l3) 

        # Update next value of y 
        y = y + (1.0 / 6.0)*(k1 + 2 * k2 + 2 * k3 + k4) 
        z = z + (1.0 / 6.0)*(l1 + 2 * l2 + 2 * l3 + l4) 

        #result.append(y)                
        t = t + h # Update next value of t
    return y 

iterations = np.arange(theta_init,theta_end+theta_step,theta_step)   # number iterations for omega sweep

start_time = time.time()
#for k in iterations:     # for serial calculation
#            rungeKutta4(k)
pool = mp.Pool(mp.cpu_count()) # Step 1: Init multiprocessing.Pool()
results = pool.map(rungeKutta4, [k for k in iterations]) # Step 2: apply pool map        
end_time = time.time()            
pool.close()  # Step 3: close
print ("The program took", end_time - start_time, "s to run")

#table = np.array(result).reshape(len(iterations),n)   # rearrange array, 1 row is const. theta0
timer = np.arange(t0,t_final,h) # time array

Answer 1

Your code seems fine, and it works for me (Ipython with python3.6.9), I've attached the results and configuration below.

Did you try to run it in a different python virtual environment (outside of Anaconda)? As it has been stated before (in Ronald's comment) other IDE's have bugs with python's multiprocessing, try run it using the command line/Ipython. I have previously encountered one even in vscode

Configuration:

Results:

Answer 2

My initial comment:

IDE's can interact with Python in unexpected ways. There are several mentions of multiprocessing misbehaving in IDE's on stackoverflow. Therefore it is probably best to test your script by directly calling Python from a shell (cmd.exe on ms-windows).

You replied:

RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

For multiprocessing to work well on ms-windows, you have to be able to import the code in your script without side effects such as starting a new process.

This means that you have to put your everything except imports and function/class definitions inside a main block, like in this exemple inspired by the documentation:

from multiprocessing import Process

def foo():
    print('hello')

if __name__ == '__main__':
    p = Process(target=foo)
    p.start()

Basically, ms-windows lacks the pretty nifty fork() system call that exists on UNIX-like systems. So the Python developers had to come up with a clever hack to make multiprocessing work on ms-windows. See e.g. this answer for the gory details.

Edit On ms-windows it seems that it is also necessary to put the worker function in its own file and import it. See e.g. here.

Answer 3

I found the solution to make it work in Anaconda.

apart from the

if __name__ == '__main__':

the function that is called (the worker) has to be in its own .py file that has to be imported at the beginning..

I put the RK4 in a RK4.py file that i imported at the beginning and not it works but that opens new problems in my case with passing the values back and forth...