0

I implemented a gridsearchcv on Azure ML as a pipeline but I keep getting an error that says "User program failed with TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker.

The exit codes of the workers are {SIGKILL(-9)}"

I tried changing the package versions but could not get it to work. The code runs well without error when I run it as a script but fails when I run it as a pipeline.

Any idea on how to fix this error?

mark_r
  • 1

1 Answers1

1

This error ... excessive memory usage causing the Operating System to kill the worker The exit codes of the workers are {SIGKILL(-9)}.

Thank you will kinsman and pnmartinez. Posting your suggestion as an answer to help other community members.

This is caused by the n_jobs = -1 which means parallelization in all CPUs (until last element of the array of cpus, hence -1).

You can try removing n_jobs = -1 in GridSearchCV and put it inside classifier or try n_jobs = 1 to deactivate the parallelization.

You can refer to Memory leak using gridsearchcv and TerminatedWorkerError

Ecstasy
  • 1,866
  • 1
  • 9
  • 17