1

I'm using a ForkJoinPool to create a pool of private threads for processing a large, single data stream. I'm not trying to use parallelism to process the stream faster, but instead to have multiple threads performing a relatively slow operation on the data records once they've been read from the stream.

The following is a test program to develop a Spliterator that would split on a concurrency value, rather than the actual workload, and assumes that the single data stream has a thread-safe getLine() method:

try (ConcurrentReader reader = new ConcurrentReader("alphabet.txt")) {
    int nThreads = 7;
    FileSpliterator spliterator = new FileSpliterator(reader, nThreads);
    Stream<String> fileStream = StreamSupport.stream(spliterator, true);

    ForkJoinPool pool = new ForkJoinPool(nThreads);
    pool.submit(() -> {
        // print and sleep for 100 milliseconds
        fileStream.parallel().forEach(s -> print(s));
    }).get();
    pool.shutdown();
}
catch (Exception e) {
    e.printStackTrace();
}

This works perfectly as a standalone Java program, producing a summary of values processed by thread like this:

Thread Summary:
             ForkJoinPool-1-worker-1 : 4
             ForkJoinPool-1-worker-2 : 3
             ForkJoinPool-1-worker-3 : 4
             ForkJoinPool-1-worker-4 : 4
             ForkJoinPool-1-worker-5 : 3
             ForkJoinPool-1-worker-6 : 4
             ForkJoinPool-1-worker-7 : 4

However, if I run the same code as part of a JUnit test, I get a summary like this:

Thread Summary:
             ForkJoinPool-1-worker-6 : 7
    ForkJoinPool.commonPool-worker-1 : 6
    ForkJoinPool.commonPool-worker-2 : 6
    ForkJoinPool.commonPool-worker-3 : 7

So my question is why it partially uses the common pool, and at most 4 threads, when running in the test suite? It looks like the stream's affinity for the ForkJoinPool is somehow being broken. Neither the JUnit test or standalone program is setting any JVM arguments.

Greg Cottman
  • 51
  • 5
  • I wondered about that, but it seems subtly different to me that it uses one thread from the ForkJoinPool that I created before switching to the common pool. The setting of java.util.concurrent.ForkJoinPool.common‌​.parallelism also has no affect on it using at most 4 threads. – Greg Cottman Jan 25 '17 at 20:01
  • As the answer in the linked question states: "... using a custom `ForkJoinPool` for stream processing is not officially supported, and when using `forEach`, the default pool parallelism is used to determine the stream spliterator behavior." Perhaps Java9 will make that support official. – AJNeufeld Jan 25 '17 at 20:06
  • That would be nice. This accidental affinity provides such a nice force-multiplier for combining customer thread pools with parallel streams that it would be a shame to leave it in its current patchy state. Thanks for your help @AJNeufeld! – Greg Cottman Jan 26 '17 at 01:30

0 Answers0