4

I want to spawn 200 threads simultaneously in Java. What I'm doing right now is running into a loop and creating 200 threads and starting them. After these 200 gets completed, I want to spawn another 200 set of threads and so on.

The gist here is that the first 200 threads I spawned need to be FINISHED before spawning the next set. I tried the code below, but its not working

for(int i=0;i<200;i++){
    Thread myThread = new Thread(runnableInstance);
    myThread.start();
}
for(int i=0;i<200;i++){
    Thread myThread = new Thread(runnableInstance);
    myThread.start();
}

Note: I have intentionally put the for loop Twice, but the desired effect I intend is not happening simply because the second for loop is executed before the first set of threads end their execution.

Please advise

bragboy
  • 34,892
  • 30
  • 114
  • 171
  • I really hope that you are using 200 just as an example for "a very big number". I can think of very few tasks that would use 200 identical threads. – Uri May 19 '10 at 17:51
  • @Uri: Yes, i want to make the question simple and clear. – bragboy May 19 '10 at 17:52
  • 2
    Be careful, 200 threads is fairly near the limit you can spawn on some versions of Linux. You should be able to spawn closer to 2000 on Windows. (Both values found through trial and error). I had to do this when I was doing a large number of synchronous SNMP gets to different devices--the CPU could easily handle a thousand outstanding requests. Once NIO came out, however, asynchronous was obviously a much better choice and could scale much higher. – Bill K May 19 '10 at 18:06
  • 1
    @Bill K: which versions of Linux/Java and why would that be so? Are you talking about last century 1.3 Java VMs? AFAIK Java on Linux uses NPTL since many years and Linux has been shown able to scale up to 100 000 NTPL threads in less than two seconds. Now there may be kernel or glibc limitation (1024 by default if I'm not mistaken), but it is trivial to recompile with higher values. I call FUD unless you give a more detailed explanation. http://java.sun.com/developer/technicalArticles/JavaTechandLinux/RedHat/ – SyntaxT3rr0r May 19 '10 at 18:31
  • 1
    @Bill K: just tried on my Linux workstation: by default, without changing any parameter, I can launch close to 4000 Java threads. A trivial recompile and I'd have that number go in the 5 digits (heck, maybe even six on my modest workstation). So, yup, I really call a *"200 threads is fairly near the limit you can spawn on some versions of Linux"* FUDdy at best. Explanations needed, because you surely aren't talkin about "regular" Linuxes/JVMs out there. I've got a hard time thinking about a Linux versions not supporting more than 200 NPTLs. – SyntaxT3rr0r May 19 '10 at 18:40
  • 1
    @WizardOfOdds Good thing, thanks. When I tried this it was on Red Hat Linux circa 199x or early 2000s, so perhaps things have changed. I revoke my warning :) It is worth noting, however, that there IS a limit and it is OS & OS configuration dependent. Also, I tended to consider about 1/2 the threads I could launch "The Limit" because after that reclaiming sometimes wasn't fast enough and sometimes I couldn't create new threads because of it. So windows was actually closer to 4k and linux closer to 400... – Bill K May 19 '10 at 20:08

6 Answers6

17

You should keep a list of the threads you have created. Then once you have started all of them, you can loop over the list and perform a join on each one. When the join loop finishes, all of your threads will have run to completion.

List<Thread> threads = new List<Thread>();
for(int i=0;i<200;i++){
    Thread myThread = new Thread(runnableInstance);
    myThread.start();
    threads.add(myThread);
}
//All threads are running
for(Thread t : threads) {
    t.join();
}
//All threads are done
unholysampler
  • 17,141
  • 7
  • 47
  • 64
8

Feels homeworky, but ...

spawnAndWait() {
 Thread T[] = new Thread[n];
 for(int i=0;i<n;i++){
     T[i] = new Thread(runnableInstance);
     T[i].start();
 }
 for (Thread t : T) {
  t.join();
 }
}
Justin
  • 4,437
  • 6
  • 32
  • 52
8

If you are using Java 1.5, use the concurrent package, if you are using Java 1.4.2, the package is still available as a backport, I am pretty sure.

That being said, I had a similar tasks to do lately; you can achieve it very easily by using ExecutorService from which you wait on the Future object to return to know whether your tasks are complete. Very clean pattern - maybe not exactly what you are trying to achieve, but in real life it works really well :-).

Some code:

BlockingQueue<Runnable> workQueue = new LinkedBlockingQueue<Runnable>();
taskExecutor = new ThreadPoolExecutor(200, 200, 30, TimeUnit.SECONDS, workQueue);

futureList.add(taskExecutor.submit(new MyCallable()));

for (Future<Object> future : futureList) {
    future.get(); // wait for the task to complete
}

//... instantiate your second group of executors
Community
  • 1
  • 1
Etienne
  • 901
  • 6
  • 15
1

You need to wait on the threads to finish by using join():

// run first set of threads
List<Thread> threads = threadsList; // save reference to all threads
for(Thread t : threads){
    t.join();
}
// all threads finished

// run other set of threads
Yuval Adam
  • 161,610
  • 92
  • 305
  • 395
  • I'm not downvoting, but your join inside the loop means that it'll wait for each thread to complete before starting the next... you've serialized those 200 threads. – CPerkins May 19 '10 at 17:52
  • You need to join after starting them. – Justin May 19 '10 at 17:52
  • CPerkins - incorrect, this snippet just iterates on the threads and `join()`s them. In anycase @unholysampler has a better snippet. – Yuval Adam May 19 '10 at 17:57
1

Sort of pseudocode, you acnt instanciate callable ;-) 

while( moreThreads ) {
    List<Callable<Integer>> threads = new ArrayList<Callable<Integer>>();
    for( int i = 0; i < 200; ++i ) {
        threads.add(new Callable<Integer>() );
    }

    FixedThreadPool pool = Executers.newFixedThreadPool(200);
    pool.invokeAll(threads);
    pool.shutdown();
}
InsertNickHere
  • 3,616
  • 3
  • 26
  • 23
1

Another solution is using Latch

final CountDownLatch latch = new CountDownLatch(200);

  for(int i=0; i<10; i++)
{

 Thread thread = new Thread()
{
    public void run()
{
     try{
         //do your work  

     }finally{
      latch.countDown();
     }
    }
   };

     }  
            //will block here util your threads all finished
  latch.await();
miz
  • 41
  • 2