Parallelize MATLAB for loop to calculate MLE

Question

I am attempting to speed up my MATLAB code by using parfor, however, I am doing it incorrectly. My code is rather simple, I am fitting some data using MATLAB's built-in mle function by using varying initial guesses for the mean (mm) and variance (vv). onestagepdf2 is my probability density function.

Here is the code snippit:

mm=linspace(.1, 1, 2); % mean
vv=linspace(.1, 2, 2); % variance
N=length(mm);
n=length(vv);

pd=zeros(n*N,2);
ld = NaN*ones(n*N,1);

options = statset('MaxIter',10000, 'MaxFunEvals',10000);

parfor i=1:N  % pick a mean
    m=mm(i);
    parfor j=1:n  %  pick a variance
        v=vv(j);
        x0=[m,v];
        [p,conf1]=mle(data,'pdf',@onestagepdf2,'start',x0, 'upperbound', [Inf Inf],'lowerbound',[0 0],'options',options)
        pd(n*(i-1)+j,:)=p;  % store parameter values from mle
        l=onestagepdf2(data,p(1),p(2)); % evaluate pdf with parameter values
        ld(n*(i-1)+j)=sum(log(l));  % store likelihood value

    end
end

The error that I receive is:

'The variable pd in a parfor cannot be classified.'

score 2 · Accepted Answer · edited May 23 '17 at 12:03

pd = zeros(n, N, 2); %initialise culprits
ld= zeros(n,N);
parfor ii=1:N  % pick a mean
    m=mm(ii);
    for jj=1:n  %  Parallellise the second parfor
        v=vv(jj);
        x0=[m,v];
        [p,conf1]=mle(data,'pdf',@onestagepdf2,'start',x0, 'upperbound', [Inf Inf],'lowerbound',[0 0],'options',options)
        pd(ii, jj, :) = p;=p;  % store parameter values from mle
        l=onestagepdf2(data,p(1),p(2)); % evaluate pdf with parameter values
        ld(ii,jj)=sum(log(l));  % store likelihood value

    end
end

Your pd was indeed the culprit, as @Trilarion stated, for the error you got. Probably ld also isn't too great, with the same syntax, so initialise that as well. This happened because parfor wants to know what the size of all variables within the loop is before executing. That this is a fixed maximum size is unknown to MATLAB, since you are using the loop variables to "change" the size.

Probably you had "orange wiggles" below those two lines (like spell-check errors) when you ran this as a for loop saying that "pd appears to be growing in size each iteration. Please consider preallocating for speed". This is required by parfor, since the order of iteration is not sequential it is impossible to grow arrays this way.

Secondly, you cannot nest parfor loops. You can use things like a function with a parfor and run that within the parfor, but that won't get you a speed-up since you are already using all your workers.

See Saving time and memory using parfor in Matlab? for more general information on parfor especially on speed.

I need to initialize `pd` and `ld` inside the `parfor` loop as opposed to outside? — user3603290, Nov 03 '15 at 10:40
hum, no. Indeed you need to initialise them outside, my bad, I'll update the post. — Adriaan, Nov 03 '15 at 10:41
`n*(i-1)+j` is not larger than `n*N` here, so the size of the variable is not growing. It's just that Matlab is not sure about it. — NoDataDumpNoContribution, Nov 03 '15 at 10:48

NoDataDumpNoContribution · Answer 2 · 2015-11-03T10:45:15.190

1

You want a sliced, output variable but Matlab is not clever enough to detect that n*(i-1)+j is actually reasonable and won't interfere with an asynchronous evaluation.

Just do it as separate dimensions

pd = zeros(n, N, 2);
...
  % in the loop
  pd(i, j, :) = p;

That will work.

Please note, that Matlab does not allow nested parfors. However, you also do not need them if N is larger than the number of workers. See also the documentation.

edited Nov 03 '15 at 10:45

answered Nov 03 '15 at 10:24

NoDataDumpNoContribution

10,591
9
64
104

The same goes for ld. – NoDataDumpNoContribution Nov 03 '15 at 10:25
@Adriaan With say 4-8 workers and N probably larger than this the asker does not need a second parfor anyway. But he still needs to expand the dimensionality of pd and ld in order to get the outer parfor to work. – NoDataDumpNoContribution Nov 03 '15 at 10:30
that seemed to fix the issue. I do have a follow up question: when I print `max(ld)`, it prints the maximum value twice, i.e., `-746.8349 -746.8349`. Could you explain why? – user3603290 Nov 03 '15 at 10:51
2

@user3603290 If you have a new question that is sufficiently different from this one then please ask [a new question](http://stackoverflow.com/questions/ask). If one of the answers in this post solved your problem then please consider [accepting it](http://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work). – IKavanagh Nov 03 '15 at 10:53

Parallelize MATLAB for loop to calculate MLE

2 Answers2