Submitting tasks to a thread-pool gives RejectedExecutionException

Question

I am working on a social game running mostly native code in Android NDK. The game has 3 main ndk pthreads:

1. a game thread
2. a server communication thread
3. the main rendering thread (called through Renderer.onRender)

Other than that, on java side, we are using AdWhirl which spawns their own thread through their own ScheduledExecutorService, but we have wrapped every call to "schedule", "submit", "post", "start" etc with try-catch block to catch the RejectedExecutionException. However, the dreadly RejectedExecutionException still occurs on every new versions we submitted.

The stack trace from Android Market leaves hardly any more clues for me and our QA department also find it difficult to pinpoint the problem as it hardly occur during test (only our users reported crash). It affects only a small portion of our users but still it is more than 7,000 crashes per week (small portion compared with high number of install base)

java.util.concurrent.RejectedExecutionException
at         java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1876)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:774)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1295)
at android.os.AsyncTask.execute(AsyncTask.java:394)
at c.onProgressUpdate(Unknown Source)
at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:432)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:123)
at android.app.ActivityThread.main(ActivityThread.java:4632)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:521)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:858)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
at dalvik.system.NativeStart.main(Native Method)

    java.util.concurrent.RejectedExecutionException: pool=128/128, queue=10/10
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1961)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:794)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1315)
at android.os.AsyncTask.execute(AsyncTask.java:394)
at c.onProgressUpdate(Unknown Source)
at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:432)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:123)
at android.app.ActivityThread.main(ActivityThread.java:3691)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:507)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:847)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:605)
at dalvik.system.NativeStart.main(Native Method)

    java.util.concurrent.RejectedExecutionException
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1876)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:774)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1295)
at android.os.AsyncTask.execute(AsyncTask.java:394)
at c.onProgressUpdate(Unknown Source)
at android.os.AsyncTask$InternalHandler.handleMessage(AsyncTask.java:432)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:123)
at android.app.ActivityThread.main(ActivityThread.java:4627)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:521)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:858)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
at dalvik.system.NativeStart.main(Native Method)

Answer 1

While you should of course try to keep things as efficient as possible, there is no arbitrary limit to the number of threads you are "allowed" to run, it all depends on how you structure your code.

The ThreadPoolExecutor class is extremely well documented, and is where the issue you're seeing is originating. I would recommend reading through it, check out

To start with I'm guessing that you're building this with Ant and aren't using these parameters on your javac node:

<javac debug="true" debuglevel="lines,vars,source" />

Either that or the obfuscator you are apparently using are the reason that what would normally be the most important part of a stack trace is instead simply outputting:

c.onProgressUpdate(Unknown Source)

This is the current ICS 4.0.4 source for ThreadPoolExecutor.AbortPolicy, as you can see it's basically a catch-all that always throws an exception:

 /**
 * A handler for rejected tasks that throws a
 * {@code RejectedExecutionException}.
 */

public static class AbortPolicy implements RejectedExecutionHandler {
    /**
     * Creates an {@code AbortPolicy}.
     */
    public AbortPolicy() { }

    /**
     * Always throws RejectedExecutionException.
     *
     * @param r the runnable task requested to be executed
     * @param e the executor attempting to execute this task
     * @throws RejectedExecutionException always.
     */
    public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
        throw new RejectedExecutionException("Task " + r.toString() +
                                             " rejected from " +
                                             e.toString());
    }
}

Additionally, you'll find the defaultHandler declared at the top of ThreadPoolExecutor:

private static final RejectedExecutionHandler defaultHandler = new AbortPolicy();

So finally, if you look at the default Constructor for ThreadPoolExecutor:

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue,
                          ThreadFactory threadFactory) {
    this(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue,
         threadFactory, defaultHandler);
}

You'll see that it's instantiating itself using it's AbortPolicy class, which is its default RejectedExecutionHandler.

ThreadPoolExecutor also includes several other RejectedExecutionHandler subclasses that you could set as the default, such as:

  /**
 * A handler for rejected tasks that silently discards the
 * rejected task.
 */
public static class DiscardPolicy implements RejectedExecutionHandler {
    /**
     * Creates a {@code DiscardPolicy}.
     */
    public DiscardPolicy() { }

    /**
     * Does nothing, which has the effect of discarding task r.
     *
     * @param r the runnable task requested to be executed
     * @param e the executor attempting to execute this task
     */
    public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
    }
}

The other 3 ThreadPoolExecutor constructors include a handler option, so you could either create an instance of it using a different handler, or create you're own subclass, similar to this:

package com.justinbuser;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.Executors;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class NoThrowThreadPool extends ThreadPoolExecutor {

    private static final RejectedExecutionHandler defaultHandler = new AdoptPolicy();

    public NoThrowThreadPool(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, Executors.defaultThreadFactory(), defaultHandler);
        setRejectedExecutionHandler(defaultHandler);
    }

    public NoThrowThreadPool(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue,
            ThreadFactory threadFactory) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, defaultHandler);
    }

    public NoThrowThreadPool(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue,
            RejectedExecutionHandler handler) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, Executors.defaultThreadFactory(), defaultHandler);
    }

    public NoThrowThreadPool(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue,
            ThreadFactory threadFactory, RejectedExecutionHandler handler) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, defaultHandler);
    }

    public static class AdoptPolicy extends ThreadPoolExecutor.AbortPolicy {

        @Override
        public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {
            new RejectedExecutionException("Task " + r.toString() + " rejected from " + e.toString()).printStackTrace();
        }
    }
}

Answer 2

You'll have to check your code, it is creating far too many AsyncTask's than it is allowed to.

The policy is set to

private static final int CORE_POOL_SIZE = 1;
private static final int MAXIMUM_POOL_SIZE = 10;
private static final int KEEP_ALIVE = 10; 

_{Note: this varies in the different versions of Android}

Answer 3

New tasks submitted in method execute(java.lang.Runnable) will be rejected when the Executor has been shut down, and also when the Executor uses finite bounds for both maximum threads and work queue capacity, and is saturated.

http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/concurrent/ThreadPoolExecutor.java