Limit number of threads in Groovy

Question

How can I limit number of threads that are being executed at the same time? Here is sample of my algorithm:

for(i = 0; i < 100000; i++) {
    Thread.start {
        // Do some work
    }
}

I would like to make sure that once number of threads in my application hits 100, algorithm will pause/wait until number of threads in the app goes below 100.

Currently "some work" takes some time to do and I end up with few thousands of threads in my app. Eventually it runs out of threads and "some work" crashes. I would like to fix it by limiting number of pools that it can use at one time.

Please let me know how to solve my issue.

score 10 · Accepted Answer · edited Nov 26 '15 at 11:53

10

I believe you are looking for a ThreadPoolExecutor in the Java Concurrency API. The idea here is that you can define a maximum number of threads in a pool and then instead of starting new Threads with a Runnable, just let the ThreadPoolExecutor take care of managing the upper limit for Threads.

Start here: http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html

import java.util.concurrent.*;
import java.util.*;

def queue = new ArrayBlockingQueue<Runnable>( 50000 )
def tPool = new ThreadPoolExecutor(5, 500, 20, TimeUnit.SECONDS, queue);

for(i = 0; i < 5000; i++) {
    tPool.execute {
      println "Blah"
    }
}

Parameters for the ThreadBlockingQueue constructor: corePoolSize (5), this is the # of threads to create and to maintain if the system is idle, maxPoolSize (500) max number of threads to create, 3rd and 4th argument states that the pool should keep idle threads around for at least 20 seconds, and the queue argument is a blocking queue that stores queued tasks.

What you'll want to play around with is the queue sizes and also how to handle rejected tasks. If you need to execute 100k tasks, you'll either have to have a queue that can hold 100k tasks, or you'll have to have a strategy for handling a rejected tasks.

edited Nov 26 '15 at 11:53

Mohamed Taher Alrefaie

15,698
9
48
66

answered Nov 20 '12 at 01:50

Tim O'Brien

9,623
5
30
36

Is it possible to get a small example? – MeIr Nov 20 '12 at 10:20
@Melr - I added an example, there you go. – Tim O'Brien Nov 20 '12 at 16:34
Strangely enough but above example doesn't behave as expected in Grails :(. Thank you for an answer, I think I might have to ask another question, specific to Grails environment. – MeIr Nov 22 '12 at 06:23
Groovy 2? That's what I used, and I verified it in the console. I'm guessing you might be using an earlier version of groovy – Tim O'Brien Nov 22 '12 at 14:14
I am developing Grails (2.0.1) project. – MeIr Nov 22 '12 at 15:07
If you are wondering how to join those threads: https://stackoverflow.com/questions/10934187/how-to-wait-for-a-threadpoolexecutor-to-finish – lepe Feb 27 '18 at 00:54

Limit number of threads in Groovy

1 Answers1