ExecutorService that executes tasks sequentially but takes threads from a pool

Question

I am trying to build an implementation of the ExecutorService, let's call it SequentialPooledExecutor, with the following properties.

1. All instances of SequentialPooledExecutor share the same thread pool

2. Calls on the same instance of SequentialPooledExecutor are executed sequentially.

In other words, the instance waits for the termination of the currently executing task before start processing the next task in its queue.

I am currently in the process of implementing the SequentialPooledExecutor myself, but I am wondering if I am reinventing the wheel. I looked into different implementations of ExecutorService, for example those that are provided by the Executors class, but I did not find one that meets my requirements.

Do you know whether there are existing implementations that I am missing, or should I continue with implementing interface myself?

EDIT:

I think my requirement are not very clear, let's see if I can explain it with other words.

Suppose I have a series of sessions, say 1000 of them (the things that I was calling instance of the executor before). I can submit tasks to a session and I want the guarantee that all tasks that are submitted to the same session are executed sequentially. However, tasks that belong to different sessions should have no dependency from each other.

I want to define an ExecutorService that executes these tasks, but uses a bounded number of threads, let's say 200, but ensures that a task is not started before the previous one in the same session is finished.

I don't know if there is anything existing that already does that, or if I should implement such an ExecutorService myself.

Answer 1

If you have thousands of keys which must be processed sequentially, but you don't have thousands of cores you can use a hashing strategy to distribute the work like this

ExecutorService[] es = // many single threaded executors

public <T> Future<T> submit(String key, Callable<T> calls) {
    int h = Math.abs(key.hashCode() % es.length);
    return es[h].submit(calls);
}

In general you only need 2 * N threads to keep N cores busy, if your task is CPU bound, more than that just adds overhead.

Answer 2

If you want to execute your task sequentially simply create a ExecutorService with only one thread thanks to Executors.newSingleThreadExecutor().

If you have different type of tasks and you want to execute only the tasks of the same type sequentially, you can use the same single threaded ExecutorService for the same type of tasks, no need to reinvent the wheel.

So let's say that you have 1 000 different type of tasks, you could use 200 single threaded ExecutorService, the only thing that you need to implement yourself is the fact that you always need to use the same single threaded ExecutorService for a given type of task.

Answer 3

private Map<Integer, CompletableFuture<Void>> sessionTasks = new HashMap<>();
private ExecutorService pool = Executors.newFixedThreadPool(200);

public void submit(int sessionId, Runnable task) {  
    if (sessionTasks.containsKey(sessionId)) {
        sessionTasks.compute(sessionId, (i, c) -> c.thenRunAsync(task, pool));
    } else {
        sessionTasks.put(sessionId, CompletableFuture.runAsync(task, pool));
    }
}

If a session has no task, a new task is created and run in the provided pool. If a session already has a tasks when a new task is added, the latter is chained (with thenRun) to the previous one, ensuring order.

Answer 4

@Nicolas's answer is probably your best bet as it is simple, well tested, and efficient.

If however it can not meet your requirement, I would do it like so :

1. Do not make "SequentialPooledExecutor" an executor service, make it a facade for a "pool" of single thread executor services
2. Make your "SequentialPooledExecutor" implement a submit method (takes a Runnable / Callable, and a String that represents a "queue name"), returns a Future, like an executor service
3. On call of this method, make your "SequentialPooledExecutor" dispatch to one of its internal, single thread, executor service, by taking the hash of the queue name, and dispatching it to the corresponding internal executor.

The hashing part that takes place at step 3 allows you to have the tasks for each "queue name" always go to the same (single thread) executor service inside of your "SequentialPooledExecutor".

Another possible route is the use of CompletionStage and CompletableFutures. These are, in effect, listenable futures (that have a completion handler). With these, the first time you have a "session", you create a CompletableFuture with your first task, and hold on to it. At each new task, you combine the previous future with the new task, calling thenAcceptAsync (or any of the like). What you get is a linear chain of execution tasks.

Answer 5

If you want to configure bounded queue, use ThreadPoolExecutor

ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, 
TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
RejectedExecutionHandler handler)

For your use case, use ThreadPoolExecutor as

ThreadPoolExecutor executor =    
ThreadPoolExecutor(1,1,60,TimeUnit.SECONDS,new ArrayBlockingQueue<Runnable>(1000));

Above code caps size of queue is ThreadPoolExecutor as 1000. If you want to use custom rejected execution handler, you can configure RejectedExeutionHandler.

Related SE question:

How to properly use Java Executor?

Answer 6

Recently I encountered the same problem. There is no built-in class for this, but a queue is close enough. My simple implementation looks like this (maybe it's helpful for others looking for examples on the same issue)

public class SerializedAsyncRunnerSimple implements Runnable {
private final ExecutorService pool;
protected final LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue<>(); //thread safe queue
protected final AtomicBoolean active = new AtomicBoolean(false);


public SerializedAsyncRunnerSimple(ExecutorService threadPool) {this.pool = threadPool;}


public void addWork(Runnable r){        
    queue.add(r);
    startExecutionIfInactive();
}

private void startExecutionIfInactive() {
    if(active.compareAndSet(false, true)) {
        pool.execute(this);
    }
}

@Override
public synchronized void run() {
    while(!queue.isEmpty()){
        queue.poll().run();
    }
    active.set(false); //all future adds will not be executed on this thread anymore
    if(!queue.isEmpty()) { //if some items were added to the queue after the last queue.poll
        startExecutionIfInactive();// trigger an execution on next thread
    }
}