2

When I use scipy.linalg.svd() to process a large matrix, the program gives the following error message:

Traceback (most recent call last):
  File "LaplacianMatrix.py", line 98, in <module>
    LaplacianInverse = LaplacianPinv(Laplacian)
  File "LaplacianMatrix.py", line 67, in LaplacianPinv
    UL,SL,VL = linalg.svd(Laplacian)
  File "/home/task3/dylan/anaconda3/lib/python3.6/site-packages/scipy/linalg/decomp_svd.py", line 125, in svd
    compute_uv=compute_uv, full_matrices=full_matrices)
  File "/home/task3/dylan/anaconda3/lib/python3.6/site-packages/scipy/linalg/lapack.py", line 712, in _compute_lwork
    raise ValueError("Too large work array required -- computation cannot "
ValueError: Too large work array required -- computation cannot be performed with standard 32-bit LAPACK.

So what can I do to make the code work well?

seralouk
  • 30,938
  • 9
  • 118
  • 133
Zhiyuan Lu
  • 21
  • 2
  • Possible duplicate of [computation cannot be performed with standard 32-bit LAPACK](https://stackoverflow.com/questions/49626572/computation-cannot-be-performed-with-standard-32-bit-lapack) – Mike Jun 20 '19 at 14:25
  • it's not a duplicate at all!!! I have exactly the same problem. I want to do SVD on a large matrix and I get the same error – seralouk Jun 20 '19 at 17:31
  • Any idea how to solve this? Here is my code: `import numpy as np; from scipy.linalg import svd: a = np.ones((30000,30000)); u,s,vh = svd(a)` and here my ticket: https://github.com/scipy/scipy/issues/10337 – seralouk Jun 20 '19 at 19:57
  • Thanks, serafeim. Your answer do solve my problem, but I do not know why this way works either. – Zhiyuan Lu Jun 21 '19 at 03:32
  • Thanks,@Mike, I have tried this way but failed with many obstacles cause I have no enough authorization on a linux server. – Zhiyuan Lu Jun 21 '19 at 03:49
  • @jean-francois why was my answer deleted? – seralouk Jun 21 '19 at 06:08
  • see my answer here: https://stackoverflow.com/questions/49626572/computation-cannot-be-performed-with-standard-32-bit-lapack/56693335#56693335 – seralouk Jun 21 '19 at 06:13

1 Answers1

0

The LAPACK C lib has 32 bit long signed integers assigned, but your error says you need to work with longer integers. See an in-depth discussion here.

For a sanity check and to make sure this is not a memory related issue try overcommiting: As sudo run echo 1 > /proc/sys/vm/overcommit_memory and then try running again your computation and see if it goes through this time. Make sure also you have enough ram for your computations to go through if you are doing everything on memory. Check also the answer of this SO post.

But your problem is the C lib.

pebox11
  • 3,377
  • 5
  • 32
  • 57