7

I'm working on an algorithm, and I've made no attempt to parallelize it other than just by using numpy/scipy. Looking at htop, sometimes the code uses all of my cores and sometimes just one. I'm considering adding parallelism to the single-threaded portions using multiprocessing or something similar.

Assuming that I have all of the parallel BLAS/MKL libraries, is there some rule of thumb that I can follow to guess whether a numpy/scipy ufunc is going to be multithreaded or not? Even better, is there some place where this is documented?

To try to figure this out, I've looked at: https://scipy.github.io/old-wiki/pages/ParallelProgramming, Python: How do you stop numpy from multithreading?, multithreaded blas in python/numpy.

mostsquares
  • 834
  • 8
  • 27
  • For future readers, the joblib maintainers have a cool package for controlling threads in blas/etc: https://github.com/joblib/threadpoolctl – mostsquares Dec 18 '19 at 20:45

3 Answers3

2

You may try to take IDP package (Intel® Distribution for Python) which contains versions of NumPy*, SciPy*, and scikit-learn* with integrated Intel® Math Kernel Library.

This would give you threading of all Lapack routines automatically whether this make sense to do. Here you find out the list of threaded mkl’s functions : https://software.intel.com/en-us/mkl-linux-developer-guide-openmp-threaded-functions-and-problems

Gennady.F
  • 571
  • 2
  • 7
  • Why did you put an asterisk next to the package names in your answer? The same is used in Intel's documentation (and others), and I have never seen it explained. – Kalo Feb 28 '23 at 12:40
1

The routines intrinsic to numpy and scipy allow single threads by default. You can change that if you so choose.

# encoding: utf-8
# module numpy.core.multiarray
# from /path/to/anaconda/lib/python3.6/site-packages/numpy/core/multiarray.cpython-36m-darwin.so
# by generator 1.145
# no doc
# no imports

# Variables with simple values

ALLOW_THREADS = 1

When compiling numpy, you can control threading by changing NPY_ALLOW_THREADS:

./core/include/numpy/ufuncobject.h:#if NPY_ALLOW_THREADS
./core/include/numpy/ndarraytypes.h:        #define NPY_ALLOW_THREADS 1

As for the external libraries, I've mostly found numpy and scipy to wrap around legacy Fortran code (QUADPACK, LAPACK, FITPACK ... so on). All the subroutines in these libraries compute on single threads.

As for the MKL dependencies, the SO posts you link to sufficiently answer the question.

newkid
  • 1,368
  • 1
  • 11
  • 27
0

please try to set the global variable OMP_NUM_THREADS. It works for my scipy and numpy. The functions I use are,

ling.inv() and A.dot(B)

Chenxin
  • 21
  • 5