I'm trying to parallelize my code with the joblib library and I'm getting a Runtime Warning. Google says, that I can ignore it without having any troubles, but the performance of the code slows down extremely, which is why I have to either solve this Runtime Warning or use another way of parallelizing my code.
I'm currently using Python 2.7 and MacOS Mojave and tried to reinstall the mentioned packages.
I'm using exactly the same code as they did here (see below).
from joblib import Parallel, delayed
import multiprocessing
inputs = range(10)
def processInput(i):
return i * i
num_cores = multiprocessing.cpu_count()
results = Parallel(n_jobs=num_cores)(delayed(processInput)(i) for i in inputs)
print(results)
The expected parallelisation should obviously be faster than without parallelization. Right now I'm getting this warning:
/Users/myname/miniconda3/lib/python2.7/site-packages/joblib/externals/loky/backend/semlock.py:217:
RuntimeWarning: semaphore are broken on OSX, release might increase its maximal value
"increase its maximal value", RuntimeWarning)
After that, the code is running, but considerably slower than without parallelization. So how can I use the joblib package without having this performance issue or how can I replace the joblib approach with some other code?
