R parallel backend: What happens when one process faces an exception?

Question

I am using foreach + %dopar% to achieve parallelism over multiple cores. I know some of the tasks will face exceptions. When an exception occurs:

Will the remaining tasks that were already parallelly started still complete?
Will the tasks that were not scheduled (I don't know if that's the correct term) be scheduled and eventually complete? If so, will it still be able to utilize all the cores?

I tried finding resources on this, but couldn't find any. Looks like I'm using the wrong keywords. If you have any resources, please direct me to them.

This may help you: https://stackoverflow.com/q/31138257/6103040 — F. Privé, Jul 02 '18 at 15:06
I personally try to implement exception handling via careful conditionals so that the parallel tasks will still be completed. — A Duv, Jul 02 '18 at 15:15

score 3 · Accepted Answer · answered Jul 02 '18 at 15:09

There is a parameter in foreach called .errorhandling, it could have the values of stop (default), remove or pass. their behaviour is like this:

stop: The function will be stopped.
remove: The result of this specific task will not be returned.
pass: The error object will be included with the results.

So addressing your specific question, if you have many task running in parallel and one of the task in one worker raised an exception, then it stop the process and will pass to the next task "scheduled" (that is because of the default value stop). The other task will continue as normal in parallel.

Please see this answer that explains better how are handled the errors in foreach and %dopar%.

I hope this will clarify a little your problem.

R parallel backend: What happens when one process faces an exception?

1 Answers1