Best way to implement a multiple producer/consumer pattern in Java 6

Question

So I have multiple steps stage 1 -> stage 2 -> stage 3 -> stage4 so in some cases the producer would be a consumer, and at each stage there are multiple producers/consumers to make use of multiple cpus. In case relevent some packets would miss out steps, i.e go straight from stage 1 to stage 4.

So I was going to have a class for each stage, sharing a BlockingQueue with the previous stage, but Ive also read that the ExecutorService works like a Producer/Consumer pattern all in one, so Im trying to go with the best abstraction.

However it seems to me that using an Executor, that the producer bit is done before they are submitted to the executor, in a sequential way which is not what I want.

Could anyone clarify please ?

I think it is better to leave this chaining and scheduling and sequencing to some SOA tool like apache service mix rather than implement our custom. But the end points can be invoked to be as multiple consumers. I think manually implementing this has too many variables and control issues. Just a thought. — r0ast3d, Nov 08 '11 at 18:08

score 1 · Answer 1 · answered Nov 08 '11 at 17:28

1

Take a look on Executors and thread pools. Here is the official tutorial: http://download.oracle.com/javase/tutorial/essential/concurrency/

answered Nov 08 '11 at 17:28

AlexR

114,158
16
130
208

score 1 · Accepted Answer · answered Nov 08 '11 at 18:13

1

Sounds like you need a java.util.concurrent.CompletionService for each stage, instead of a BlockingQueue.

answered Nov 08 '11 at 18:13

artbristol

32,010
5
70
103

Yeah Ive come to that my come conclusion myself – Paul Taylor Nov 09 '11 at 11:50

Best way to implement a multiple producer/consumer pattern in Java 6

2 Answers2