3

I am trying to find all non-zeros elements in a large sparse matrix K(19683000x19683000). I used find to first return the linear indices.

val = K(K~=0);
inds = find(K~=0);
[j, i] = ind2sub(size(K), inds);
% [j, i] = find(K~=0);
i = full(i);
j = full(j);

This gave some error:

Error using find
Matrix is too large to return linear indices.
Use [i,j] = find(S) for sparse matrix.
Error in (line 82)
inds = find(K~=0);
Error in run (line 64)
evalin('caller', [script ';']);

Any idea what is happening and how I can avoid it?

colddie
  • 1,029
  • 1
  • 15
  • 28
  • I have never found that error, probably because I never used matrices so large. The error is likely related with the fact that `double` variables can exactly represent integers up to `2^53-1` only. The number of elements of your matrix does not exceed that limit, but it is near – Luis Mendo Dec 29 '18 at 19:15
  • Maybe you can sidestep the issue (up to `2^53-1`) by getting the row and column indices, and then computing the linear index from those manually: `[ii,jj] = find(K); inds = ii + (jj-1)*size(K,1)` – Luis Mendo Dec 29 '18 at 19:19
  • @LuisMendo indeed this is a solution for me. Thanks. Maybe you can put your suggestion as an answer so I can accept it? – colddie Jan 02 '19 at 15:18
  • Glad it worked. I rephrased a bit and added as an answer – Luis Mendo Jan 03 '19 at 00:58

1 Answers1

3

I am not familiar with that error (probably because I never used matrices so large). The error is likely related with the fact that double variables can exactly represent integers up to 2^53 only. The number of elements of your matrix does not exceed that limit, but it is near.

Maybe you can sidestep the issue, up to 2^53, by getting the row and column indices and then computing the linear index from those manually:

[ii,jj] = find(K);
inds = ii + (jj-1)*size(K,1);

If needed, you can push the limit up to 2^64 by using a uint64 linear index instead of double:

[ii,jj] = find(K);
inds = uint64(ii) + (uint64(jj)-1)*size(K,1);
Luis Mendo
  • 110,752
  • 13
  • 76
  • 147