Rate limiting the number of threads in an Executor (Java)

Question

How can the number of threads executing in an Executor be limited? For example, in the code bellow:

            executor = Executors.newFixedThreadPool(this.noOfThreads);
            for (int i = 0; i < sections.size(); i++) {
                DomNode section = sections.get(i);
                // Retrieve the url of the subsection
                String subsectionUrl = section.getNodeValue();

                if(i != 0 && (i % noOfThreadsPerSection == 0)) {
                    // Add section threads to the executor
                    executor.execute(new BrowseSection(filter, webClient, subsectionUrl));
                } else {
                    executor.shutdown();
                    try {
                        executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
                    } catch (InterruptedException e) {
                        // We should normally never get here
                    }
                }
            }

when executor.execute() is called, what happens if the thread pool is filled with running threads?

The code above does not check whether threads are still running in the executor, but rather the number of threads that have been launched.

in your example, the number of threads is limited by this.noOfThreads. it will never be greater than this — Jérémie B, Feb 07 '16 at 16:35
noOfThreads is the size of the batch of threads running at a given time (provided that there are more sections). But if the number of sections is greater than this.noOfThreads, and the threads are still not finished exectuing what happens when the if is reached? Does the executor wait until some of the threads are finished or not? — Sebi, Feb 07 '16 at 16:37
You can read more about ThreadPoolExecutor: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ThreadPoolExecutor.html — Jean Logeart, Feb 07 '16 at 16:37
@Jérémie B Thanks. But is it possible to fill the executor with a new thread is finished, and not the whole batch of this.noOfThreads? — Sebi, Feb 07 '16 at 16:43

Peter Lawrey · Accepted Answer · 2016-02-08T01:42:11.487

2

The pool creates a fixed number of threads whether there is anything to do or not. It doesn't change based on how you use it.

Your jobs are passed to a queue and the thread pool gets tasks from the queue. When the pool is not all busy, tasks are pulled off the queue almost as soon as it is added. When you have more tasks, the queue gets longer.

In Java 8, it avoids creating a task for each item but instead breaks down the work into sections. This way it can be more efficient than using an ExecutorService in a naive way.

In Java 8 you would write.

sections.parallelStream()
        .map(s -> s.getNodeValue())
        .forEach(s -> new BrowseSection(filter, webClient, s).run());

There is no need to startup the thread pool and shut it down again etc. This waits until all tasks have completed.

You can change the size of the ForkJoinPool.commonPool() by setting on the command line

-Djava.util.concurrent.ForkJoinPool.common.parallelism=N

You can also set this pragmatically provided you do it before this class loads.

edited Feb 08 '16 at 01:42

answered Feb 07 '16 at 17:27

Peter Lawrey

525,659
79
751
1,130

1

Shouldn't the last parameter subsectionUrl be replaced with s. – Sebi Feb 07 '16 at 18:50
1

@Sebi indeed, I have fixed it now. – Peter Lawrey Feb 07 '16 at 18:52
And also, how does the Executor come into play? This code creates a thread for each entry in s and I want to limit that, set an upper bound to the number of threads the app uses. – Sebi Feb 07 '16 at 18:57
How is it possible to control the number of threads through the lambda expression? – Sebi Feb 07 '16 at 19:21
@Sebi yes, though by default it uses all the cpus on your machine. I have updated my answer. – Peter Lawrey Feb 08 '16 at 01:46
This answer works but would it be possible to control the thread pool size from the code? In my case I have two functions, one that browses sections in the forum and the other browsers a number of threads inside the section that the former browses. If I use this approach the first function (section browser) will take up more threads than the "forum thread" browser. – Sebi Feb 10 '16 at 13:19
1

@Sebi what would be the downside of using all the cpus on the machine. I design such systems so using limited thread pools can be useful but I suspect you don't need this level of control in this case. – Peter Lawrey Feb 10 '16 at 13:35
I'm planning on using a heuristic model to filter post content. I do not know as of yet how will it scale if all cores are used (I don't want to freeze the end user's machine). My idea is to allow the user to set the number of threads. – Sebi Feb 10 '16 at 13:44
And the app also retrieves web pages from an online forum concurrently. Using too many threads may also impact the server's performance. – Sebi Feb 10 '16 at 13:54
@Sebi this is true though the default pool won't create more threads than you have CPUs. If you have another pool for IO, this shouldn't impact the server if they send most of their time waiting. – Peter Lawrey Feb 10 '16 at 15:08

score 1 · Answer 2 · answered Feb 07 '16 at 16:38

From the java documentation newFixedThreadPool has an unbounded queue and so all the other threads will wait:

Creates a thread pool that reuses a fixed number of threads operating off a shared unbounded queue. At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks. The threads in the pool will exist until it is explicitly shutdown.

score 1 · Answer 3 · edited May 23 '17 at 11:52

Executors & ExecutorService does not provide APIs to control task queue size, which is unbounded by default.

If your tasks take shorter time intervals, using ExecutorService is OK.

Prefer ThreadPoolExecutor, which provides better control of TaskQueueSize, Rejection Handling mechanism.

ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, 
TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
RejectedExecutionHandler handler)

You can control thread pool size dynamically with below API

setCorePoolSize(newLimit)

Have a look at these SE question for more details:

Dynamic Thread Pool

Rate limiting the number of threads in an Executor (Java)

3 Answers3