Repeatedly use a threadpool in JAVA

Question

I'm calling an API to get a list of IDs. This list contains around 55.000 IDs. Then I wish to initiate threads and based on the ID, get information about its object and store it in a database. I've managed to do this using threads but would like to know how to optimize this using threadpools.

What I want is to make 10 threads, add them to a threadpool, wait for them to finish, wait 10 seconds (or I'll get 429, too many requests) and then start 10 more until all are done. What needs to be done in order to do this?

Also the list is of unknown length so it might not be 10 items in the last pool, if this makes any difference.

This is what I've got working so far. (sleep not really working, after a while 429 happens anyway)

JSONObject IDs = getIDs();

    for (int i = 0; i <= IDs.length(); i++) {
        try {
            int ID = IDs.getJSONArray("app").getInt("appid");
            // get info on object and store it in database
            Thread t = new Thread(new MyRunnable(ID));
            t.start();
            // sleep or 429, too many requests
            Thread.sleep(1000);

        } catch (JSONException | InterruptedException e) {
            e.printStackTrace();
        }
    }

Learn proper basic recommended syntax as well for Java. Variable names like "IDs" is not recommended — jarvo69, Oct 28 '17 at 09:10

score 0 · Accepted Answer · answered Oct 28 '17 at 09:20

You can use CyclicBarrier for this purpose.

A synchronization aid that allows a set of threads to all wait for each other to reach a common barrier point. CyclicBarriers are useful in programs involving a fixed sized party of threads that must occasionally wait for each other. The barrier is called cyclic because it can be re-used after the waiting threads are released.

Repeatedly use a threadpool in JAVA

1 Answers1