2

I am currently trying to speed up my large sparse (scipy) matrix multiplications. I have successfully linked my numpy installation with OpenBLAS and henceforth, also scipy. I have run these tests with success.

When I use numpy.dot(X,Y) I can clearly see performance boosts and also that multiple cores are used simultaneously. However, when I use scipy's dot functionality, no such performance boosts can be seen and still one one core is used. For example:

x = scipy.sparse.csr_matrix(numpy.random.random((1000,1000)))
x.dot(x.T)

Does anyone know how I can make BLAS also work with scipy's dot functionality?

ali_m
  • 71,714
  • 23
  • 223
  • 298
fsociety
  • 1,791
  • 4
  • 22
  • 32

2 Answers2

4

BLAS is just used for dense floating-point matrices. Matrix multiplication of a scipy.sparse.csr_matrix is done using pure C++ functions that don't make any calls to external BLAS libraries.

For example, matrix-matrix multiplication is implemented here, in csr_matmat_pass_1 and csr_matmat_pass_2.

Optimised BLAS libraries are highly tuned to make efficient use of CPU caches by decomposing the dense input matrices into smaller block matrices in order to achieve better locality-of-reference. My understanding is that this strategy can't be easily applied to sparse matrices, where the non-zero elements may be arbitrarily distributed within the matrix.

ali_m
  • 71,714
  • 23
  • 223
  • 298
  • Makes sense, thanks. Do you know of any other way to speed up the matrix multiplication of sparse matrices? I also run into memory issues. – fsociety Aug 03 '14 at 11:24
  • There might be, depending on your specific application. Do your matrices have some known sparsity structure, e.g. diagonal, banded, block? Does the output matrix have a known sparsity structure? I suggest you post another question, where you give as much information as you can about what your matrices are like, and what you plan on doing with the output. – ali_m Aug 03 '14 at 11:53
  • Made a new question: http://stackoverflow.com/questions/25104733/how-to-efficiently-calculate-huge-matrix-multiplication-tfidf-features-in-pyth – fsociety Aug 03 '14 at 12:40
1

Probably too late to help you, but it may help someone else:

SciPy implementation of sparse matrix - sparse matrix multiplication is single-threaded only.

However, using https://pypi.org/project/sparse-dot-mkl/, your computation can run multithreaded, because the library uses intel MKL.

Martin
  • 21
  • 3