1

Looking for a solution to optimise processing of workloads, which are coming either from background or from API task sources. Task code is just the single Callable class. I guess that having two separate thread pools for different task sources is less elegant solution. So i need the following properties that ExecutorService should have

  • dynamically dynamically adjust task execution order based on Comparable interface. java.util.concurrent.PriorityBlockingQueue does that already. I can pass it as argument to java.util.concurrent.ThreadPoolExecutor, so that's already handled.
  • number of threads is preferred to scale up and down dynamically (c)

As long as Java community is dynamic, maybe somebody did already solve this problem into any available library? Or should I override PriorityBlockingQueue methods?

Claus Ibsen
  • 56,060
  • 7
  • 50
  • 65
nefo_x
  • 3,050
  • 4
  • 27
  • 40
  • 1
    You can get a limited form of auto-scaling using [this pattern](http://stackoverflow.com/a/15032613/552759). Also, do you care about starvation of lower priority tasks, cause that makes things "more complicated". – jtahlborn Nov 11 '14 at 20:23
  • @jtahlborn thanks, any comments are welcome – nefo_x Nov 11 '14 at 20:25
  • 1
    @nefo_x What do your jobs (Callables) look like? Are they I/O heavy or just cpu bound? If they are cpu bound, you can only scale up to the number of cores anyway. – Rafal G. Nov 11 '14 at 20:36
  • @R4J they have disk only IO – nefo_x Nov 11 '14 at 20:44

1 Answers1

1

For the dynamically scaling part, this may help a bit: https://stackoverflow.com/a/19538899/999043 (actually, you want to read the other answers, too). For the priority part, make sure sorting the queue and inserting does not impact scalability.

Community
  • 1
  • 1
Ralf H
  • 1,392
  • 1
  • 9
  • 17