2

I have 2 nested loops which do the following:

  • Get two rows of a matrix
  • Check if indices meet a condition or not
  • If they do: calculate xcorr between the two rows and put it into new vector
  • Find the index of the maximum value of sub vector and replace element of LAG matrix with this value

I dont know how I can speed this code up by vectorizing or otherwise.

b=size(data,1);
F=size(data,2);
LAG= zeros(b,b);  

for i=1:b
    for j=1:b
        if j>i
            x=data(i,:);
            y=data(j,:);
            d=xcorr(x,y);
            d=d(:,F:(2*F)-1); 
            [M,I] = max(d);
            LAG(i,j)=I-1;
            d=xcorr(y,x);
            d=d(:,F:(2*F)-1);
            [M,I] = max(d);
            LAG(j,i)=I-1;
        end
    end
end
Wolfie
  • 27,562
  • 7
  • 28
  • 55

1 Answers1

1

First, a note on floating point precision...

You mention in a comment that your data contains the integers 0, 1, and 2. You would therefore expect a cross-correlation to give integer results. However, since the calculation is being done in double-precision, there appears to be some floating-point error introduced. This error can cause the results to be ever so slightly larger or smaller than integer values.

Since your calculations involve looking for the location of the maxima, then you could get slightly different results if there are repeated maximal integer values with added precision errors. For example, let's say you expect the value 10 to be the maximum and appear in indices 2 and 4 of a vector d. You might calculate d one way and get d(2) = 10 and d(4) = 10.00000000000001, with some added precision error. The maximum would therefore be located in index 4. If you use a different method to calculate d, you might get d(2) = 10 and d(4) = 9.99999999999999, with the error going in the opposite direction, causing the maximum to be located in index 2.

The solution? Round your cross-correlation data first:

d = round(xcorr(x, y));

This will eliminate the floating-point errors and give you the integer results you expect.

Now, on to the actual solutions...


Solution 1: Non-loop option

You can pass a matrix to xcorr and it will perform the cross-correlation for every pairwise combination of columns. Using this, you can forego your loops altogether like so:

d = round(xcorr(data.'));
[~, I] = max(d(F:(2*F)-1,:), [], 1);
LAG = reshape(I-1, b, b).';


Solution 2: Improved loop option

There are limits to how large data can be for the above solution, since it will produce large intermediate and output variables that can exceed the maximum array size available. In such a case for loops may be unavoidable, but you can improve upon the for-loop solution above. Specifically, you can compute the cross-correlation once for a pair (x, y), then just flip the result for the pair (y, x):

% Loop over rows:
for row = 1:b

  % Loop over upper matrix triangle:
  for col = (row+1):b

    % Cross-correlation for upper triangle:
    d = round(xcorr(data(row, :), data(col, :)));
    [~, I] = max(d(:, F:(2*F)-1));
    LAG(row, col) = I-1;

    % Cross-correlation for lower triangle:
    d = fliplr(d);
    [~, I] = max(d(:, F:(2*F)-1));
    LAG(col, row) = I-1;

  end

end
gnovice
  • 125,304
  • 15
  • 256
  • 359
  • thanks a lot for your care:) I use this now ,but I got this error('Error using bsxfun Requested 1024x4000x4000 (244.1GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.') –  Dec 14 '17 at 15:41
  • @FaribaJangjoo: I'm guessing `data` is 4000-by-1024? That apparently produces intermediate and output variables (like `d`) that are too big. Given the size of your problem, you may be limited to using loops to save on memory. However, there are improvements you can make to your loop. I'll update shortly. – gnovice Dec 14 '17 at 15:47
  • my main matrix is 2500*412000. but I run it for 4000*400 now to check it and it gives me this error. –  Dec 14 '17 at 15:50
  • I checked my previous result with this one ,but It doesn't give me the same result as before, :( –  Dec 14 '17 at 16:18
  • @FaribaJangjoo: Try now. I removed the auto-correlation part since you aren't performing those in your code. – gnovice Dec 14 '17 at 16:22
  • it still gives me wrong result,i think in your code it doesn't consider whole rows ,I don't know if I understand it right or not... –  Dec 14 '17 at 16:27
  • @FaribaJangjoo: I'm getting matching results when I test the solutions with random data. I tried to speed it up by computing a single cross-correlation, then just flipping that result for the cross-correlation with flipped inputs. This *should* give the same result, so I don't know where the error is coming from. – gnovice Dec 14 '17 at 16:35
  • hmm:(...would you mind checking it again for another matrix? ireally need this to work...sorry for interrupting you –  Dec 14 '17 at 16:43
  • @FaribaJangjoo: I've already tested it on half a dozen random matrices and gotten matching results. Are you redefining `b` and `F` for each new matrix you test? – gnovice Dec 14 '17 at 16:49
  • I put these 3lines once above your code (b=size(data,1); F=size(data,2);LAG= zeros(b,b);)...should I do anything else? –  Dec 14 '17 at 16:51
  • i changed this line(d = fliplr(d);)with( d = xcorr(data(col, :), data(row, :));) & it gives the same results..is it ok?but its so much slower than my original code :( –  Dec 14 '17 at 17:03
  • @FaribaJangjoo: Changing that line should give you something nearly identical to your original code. What sort of data do you have? – gnovice Dec 14 '17 at 17:19
  • @FaribaJangjoo: It turns out the reason the solutions were giving different results is due to floating-point precision error, which I describe above. You have to round the cross-correlation results in your solution to avoid this. – gnovice Dec 14 '17 at 20:08
  • thanks a lot for helping me but,i use edited form ,but it still gives me a different result from what i want... –  Dec 15 '17 at 10:46
  • @FaribaJangjoo: If "what you want" is what is in the question, then what you want isn't quite right. Note that, as I discuss above, you would need to round the results you get from `xcorr` in your solution to remove floating-point errors. The results should match after you do that. – gnovice Dec 18 '17 at 18:42