4

I wanted to use custom ThreadPool for parallelStream. Reason being I wanted to use MDCContext in the task. This is the code I wrote to use the custom ThreadPool:

final ExecutorService mdcPool = MDCExecutors.newCachedThreadPool();
mdcPool.submit(() -> ruleset.getOperationList().parallelStream().forEach(operation -> {
                log.info("Sample log line");
});

When the MDC context was not getting copied to the task, I looked at the logs. These are the logs I found. The first log is executed in "(pool-16-thread-1)" but other tasks are getting executed on "ForkJoinPool.commonPool-worker". The first log also has MdcContextID. But as I am using custom ThreadPool for submitting the task, all tasks should be executing in custom ThreadPool.

16 Oct 2018 12:46:58,298 [INFO] 8fcfa6ee-d141-11e8-b84a-7da6cd73aa0b (pool-16-thread-1) com.amazon.rss.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 12:46:58,298 [INFO]  (ForkJoinPool.commonPool-worker-11) com.amazon.rss.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 12:46:58,298 [INFO]  (ForkJoinPool.commonPool-worker-4) com.amazon.rss.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 12:46:58,298 [INFO]  (ForkJoinPool.commonPool-worker-13) com.amazon.rss.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 12:46:58,298 [INFO]  (ForkJoinPool.commonPool-worker-9) com.amazon.rss.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 12:46:58,299 [INFO]  (ForkJoinPool.commonPool-worker-2) com.amazon.rss.activity.business.VariablesEvaluator: Sample log line
16 Oct 2018 12:46:58,299 [INFO]  (ForkJoinPool.commonPool-worker-15) com.amazon.rss.activity.business.VariablesEvaluator: Sample log line

Is this supposed to happen or am I missing something?

Naman
  • 27,789
  • 26
  • 218
  • 353
Akhil
  • 533
  • 2
  • 11
  • 26

1 Answers1

2

There is no support for running a parallel stream in a custom thread pool. It happens to be executed in a different Fork/Join pool when the operation is initiated in a worker thread of a different Fork/Join pool, but that does not seem to be a planned feature, as the Stream implementation code will still use artifacts of the common pool internally in some operations then.

In your case, it seems that the ExecutorService returned by MDCExecutors.newCachedThreadPool() is not a Fork/Join pool, so it does not exhibit this undocumented behavior at all.

There is a feature request, JDK-8032512, regarding more thread control. It’s open and, as far as I can see, without much activity.

Holger
  • 285,553
  • 42
  • 434
  • 765
  • me very confused right now :| what about [this](https://stackoverflow.com/a/22269778/1059372)? Or you mean it *has* to be another `ForkJoinPool`? – Eugene Oct 16 '18 at 15:14
  • 1
    @Eugene as said, it *happens* to work this way (when the custom pool is a Fork/Join pool), as the result of an implementation detail, not a planned feature. Compare with [this answer](https://stackoverflow.com/a/29272776/2711488) where Stuart Marks explicitly states that this trick is not guaranteed to work and also describes one of the issues I meant with “the Stream implementation code will still use artifacts of the common pool internally”. – Holger Oct 16 '18 at 15:20
  • 1
    this somehow impacts me (almost directly) quite a lot, it seems. I wrote a few places where I use a custom parallel stream to run in a separate ForkJoinPool as I don't want to be blocked in case someone else uses the default one, it seems that I need to reconsider the code, but I might be out of alternatives. thank you for the comment – Eugene Oct 16 '18 at 15:25