1

I m trying to multiply 300000 x 300000 sparse matrix with a single array of 300000, and its taking 7-8 seconds by matmul numpy.

NumPy matmul is what I have used. Is there any other option?

Saurabh Agarwal
  • 21
  • 2
  • You aren't using `np.matmul`, are you? `M@arr` uses `Ms` own version of matrix multiplication. – hpaulj Jan 19 '23 at 17:00
  • yes i m using A@B – Saurabh Agarwal Jan 19 '23 at 17:01
  • Is np.matmul and @ are two different operations – Saurabh Agarwal Jan 19 '23 at 17:01
  • 1
    `A@B` is translated to `A.__matmul__(B)`. `np.matmul(A,B)` first makes sure all arguments are `numpy.ndarray`, which is not what you want for a sparse matrix. Anyways, the sparse matrix multiplication is pretty well developed, so I wouldn't expect to find a faster alternative. If `A` is already `ndarray`, `A@B` may be faster, but that's not an option with your sizes. – hpaulj Jan 19 '23 at 17:07
  • Yes, actually A matrix is a sparse matrix with float32 values, and u is an array – Saurabh Agarwal Jan 19 '23 at 17:16
  • Details are missing to *make this reproducible*. For example, what kind of sparse matrix do you use (eg. CSR, DOK, etc), what datatype, what is the ratio of non-zeros values in the sparce matrix, what processor, do you have enough RAM, etc. Results can be very different regarding theses things. Thus, please provide a [minimal-reproducible-example](https://stackoverflow.com/help/minimal-reproducible-example). – Jérôme Richard Jan 19 '23 at 18:12
  • I m using CSR sparse matrix, using M1 processor 16GB ram. – Saurabh Agarwal Jan 19 '23 at 18:24
  • and ratio of non zero value is about 1% only. – Saurabh Agarwal Jan 19 '23 at 18:24
  • This matrix is already quite big. It should take 6.7 GiB. Thus, the time is not so surprising to me. That being said, I expected this to be faster. I spent a lot of time trying to generate a random matrix like this without filling my whole memory (also 16 GiB). In the end, it takes a bit more than 8 GiB on my machine and 1.3 second for the matrix multiplication. This is not optimal but reasonable. The first time, my RAM was saturated and it took 4-6 seconds. Since the M1 should be a bit faster than my CPU for that and you do not have more RAM, I think you run out of memory and use the swap. – Jérôme Richard Jan 19 '23 at 22:14

0 Answers0