1

I have a ThreadPoolExecutor that is constructed with an unbounded queue (LinkedBlockingQueue) and a core and max pool size set to the number of cpus (say 4).

Every great while I get a RejectedExecutionException. The Executor is in a running state. My understanding is this should not happen with an unbounded queue.

I haven't been able to catch this in a debugger to see exactly what is happening, but from the stack trace it looks like in ThreadPoolExecutor.execute, workQueue.offer is returning false, so it jumps to the bit where it tries to spin up a new thread. But poolSize is already at max, so it throws the rejected execution exception.

I don't quite understand this.

But regardless, should I make the max pool size a bit bigger than the core pool size?

Gray
  • 115,027
  • 24
  • 293
  • 354
marathon
  • 7,881
  • 17
  • 74
  • 137

2 Answers2

2

LinkedBlockingQueue.offer() returns false when it is at its capacity. If the capacity is not specified, then Integer.MAX_VALUE is used.

Does this happen when adding 2147483647 more tasks than your pool size?

Stephen Denne
  • 36,219
  • 10
  • 45
  • 60
  • yes, we may be maxing out the queue. I didn't think this would be a possibility, but now I am. Nothing else would make sense. Thanks. – marathon Sep 01 '11 at 04:36
2
  1. Even an "unbounded" LinkedBlockingQueue is actually bounded at Integer.MAX_VALUE. Yes, it's improbable that you're hitting that limit, but "once you eliminate the impossible...".
  2. How certain are you that the executor is still in RUNNING state? In my source code, the check for state and the offer() call are both on the same line, so you wouldn't be able to distinguish them in a stack trace.
Ryan Stewart
  • 126,015
  • 21
  • 180
  • 199
  • While I agree that a non RUNNING state would do this, the line with the state check and offer() call wouldn't be in the stack trace at all. The stack trace would include the reject(command) line. – Stephen Denne Aug 31 '11 at 03:03
  • I'm beginning to think that we are actually filling the queue. This app is running in a 64-bit VM with virtually unlimited memory available. The load is tremendous and if the worker threads stalled for whatever reason, we could probably fill this queue in a few hours. I'm going to instrument the LinkedBlockingQueue to see if this is what is going on. Thanks. – marathon Sep 01 '11 at 04:33