Java ForkJoinPool thread limit or Java stream alternative?

Question

I understand that java parallelStream(or ForkJoinPool) is designed to maximize CPU Utilization. Because of the great combination of java stream functional interfaces (such as map, reduce, filter) and ForkJoinPool I use java parallelstream and ForkJoinPool.

The problem is that ForkJoin does not limit the number of active thread. If some threads blocked while it hold large memory, ForkJoinPool tries to create more threads to meet the parallelism(running thread).

In this case cpu utilization will be maximized but heap memory will be Exhausted or even go OOM.

How can I limit the number of active threads of ForkJoinPool?

Or is there any java stream interface alternatives?

Answer 1

I understand that java parallelStream(or ForkJoinPool) is designed to maximize CPU Utilization.

That's not exactly the goal. It may have the effect of maximizing CPU utilization, but the goal is to speed up the computation. That is NOT the same thing.

How can I limit the number of active threads of ForkJoinPool?

According to this mailing list thread, one way prevent the forkjoin thread pool from exploding in pathological situations is to supply a custom ForkJoinThreadFactory that keeps track of the number of extant threads (somehow) and returns null when too many threads already exist.

Be aware that if you do hit the limit (imposed by your factory), you will get RejectedExecutionExceptions on task submission.

---

Or is there any java stream interface alternatives?

I'm not aware of one.

But I'm not convinced that you would encounter this problem at all when using Stream.parallelStream() in the normal way.

If you do encounter it and rejected executions are problematic, you probably need to look for another way to express the computation; e.g. using coroutines rather than threads, or with work queues and a Executor, or something else that I haven't thought of :-)