0

If a parfor reckons that the computer will not have enough ram to run the code in parallel will it automatically serialize it? That definitely seems to be the case.

I have two identical parfor loops (except regarding the size of the matrices within them). On the first one it easily reaches 100% CPU and half my RAM, on the second one it reaches 12-20% CPU and all my RAM, and the codes are exactly equal (except for the size of the matrices inside them).

AlessioX
  • 3,167
  • 6
  • 24
  • 40
phdstudent
  • 1,060
  • 20
  • 41
  • If your code is short enough for a [mcve], please [edit] your question to contain the code. In my experience `parfor` always throws me a simple Out of Memory error when it's out of RAM, and does noet go serial. – Adriaan Jul 18 '16 at 10:09

1 Answers1

0

I have addressed the same issue in this question here.

In short, being each worker in the Matlab pool independent from the others, each worker needs his own amount of memory.

And no, Matlab does not automatically serialise your for-loop if it goes out-of-memory. If Matlab throws a proper error (as my knowledge it does happen on Windows PCs) you can do some sort of try-catch statement. The try-catch simply tries to execute the code in the try branch and if some error happens it executes automatically the catch block. In your case it'll be something like

try
    % parfor here
catch
    % standard for here
end
Community
  • 1
  • 1
AlessioX
  • 3,167
  • 6
  • 24
  • 40
  • 1
    You can do that. I'd add the information about how bad an idea a try catch is in general, as it makes the code way slower. – Ander Biguri Jul 18 '16 at 10:24
  • 2
    The problem with a `try/catch` is that if your code runs out of memory after 2 hours of processing in parallel, you have just wasted 2 hours of time trying to do something which couldn't be completed. Better would be to estimate the size of the outputs and see whether that'd fit in the total RAM and decide based on that. – Adriaan Jul 18 '16 at 10:39