Logical indexing in matlab - need help to make faster

Question

This is what I am trying to do, created a random array to demonstrate:

% all IDs

 all_IDS = 1:216000000;    

% Array 1

X = round(1550*rand(216000000,1));
Y = round(1550*rand(216000000,1));
Z = round(90*rand(216000000,1));

% Array 2

Xsub = round(1550*rand(160000,1));
Ysub = round(1550*rand(160000,1));
Zsub = round(90*rand(160000,1));

del_val =1;

% required o/p
reqd_op = zeros(1,10); 

% boolean indexing
indx =1;
for jj = 1:160000

    VID_X = Xsub(jj);
    VID_Y = Ysub(jj);
    VID_Z = Zsub(jj);

    I2 = (X>VID_X-del_val & X<VID_X+del_val)& (Y>VID_Y-del_val & Y<VID_Y+del_val) & (Z>VID_Z-del_val & Z<VID_Z+del_val);

    len = numel(all_IDS(I2));

    reqd_op(1,indx:indx+len-1) =  all_IDS(I2);
    indx=indx+len;
end

The above code takes a lot of time as I am dealing with a very large array , Is there a way to eliminate the for loop, meaning, instead of doing Boolean indexing element by element - can I do it for the whole array at once ?

See [What is an XY-problem and how to avoid it](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem) — Sardar Usama, Aug 22 '17 at 21:10
See [Linear indexing, logical indexing, and all that](https://stackoverflow.com/q/32379805/5211833) — Adriaan, Aug 22 '17 at 21:23
btw what your loop only retains at the end is the last iteration of `jj` which is `I2 = (X>Xsub(160000)-del_val & XYsub(160000)-del_val & YZsub(160000)-del_val & Z — Sardar Usama, Aug 22 '17 at 21:25
I do take I2 and retrieve output in some form , sorry for not including it — Mechanician, Aug 22 '17 at 21:28
The answer is going to depend on your "further processing not shown here", i.e. what problem Y you are trying to solve, not necessarily the approach X you have chosen and described. — jez, Aug 22 '17 at 21:29
I apologize for that, I shall try to be precise from next time. But I have modified my code to be minimal /complete & verifiable. — Mechanician, Aug 22 '17 at 21:39
Wouldn't doing the whole array at once require building a 35GB array? That seems big. — beaker, Aug 22 '17 at 23:32

score 1 · Answer 1 · answered Aug 23 '17 at 02:41

This will run x2.5 faster, anyway, array is too big so it still takes 0.3s per loop, so 160000 loops is like 13 hours on single cpu.

if ~exist('X','var')
    % Array 1
    X = round(1550*rand(216000000,1,'single'));
    Y = round(1550*rand(216000000,1,'single'));
    Z = round(90*rand(216000000,1,'single'));
    % Array 2
    Xsub = round(1550*rand(160000,1,'single'));
    Ysub = round(1550*rand(160000,1,'single'));
    Zsub = round(90*rand(160000,1,'single'));
end

del_val =single(1);
reqd_op = zeros(1,10,'single');% required o/p

tic
index =1;
for jj = 1:10
    VID_X = Xsub(jj);
    VID_Y = Ysub(jj);
    VID_Z = Zsub(jj);
    IdxFinal=[];
    Idx1=find(abs(X-VID_X)<del_val); %little better than X>VID_X-del_val & X<VID_X+del_val)
    if ~isempty(Idx1)
        Idx2 = Idx1(Y(Idx1)>VID_Y-del_val & Y(Idx1)<VID_Y+del_val);
        if ~isempty(Idx2)
            Idx3= Idx2(Z(Idx2)>VID_Z-del_val & Z(Idx2)<VID_Z+del_val);
            IdxFinal=Idx3;
        end
    end
    len = length(IdxFinal);
    index=index+len;
    if len>0
        reqd_op(1,index:index+len-1) = IdxFinal;
    end
end
toc

Logical indexing in matlab - need help to make faster

1 Answers1