3

I have N tasks, I want them to be processed in parallel with N threads. I want to wait until all tasks are finished, store results, and then run next N tasks (and so on in a loop).

Which abstractions from java util concurrency can help me here?

I looked at ExecutorService.invokeAll(), but it returns a list of futures, so I should iterate all of them in a loop until all of them are done.

I thought, there should be more straightforward way to calculate a set of tasks.

Roman
  • 64,384
  • 92
  • 238
  • 332
  • May be this helps: http://stackoverflow.com/questions/2248131/handling-exceptions-from-java-executorservice-tasks – kosa Jun 21 '13 at 15:21

4 Answers4

6

ExecutorService.invokeAll() is probably the most straightforward way of doing it.

You don't iterate over any of them to await completion. invokeAll only returns when all of them are completed. So by the time you can iterate over them, get will return immediately.

John Vint
  • 39,695
  • 7
  • 78
  • 108
  • 1
    +1. It may be good to mention how to handle results properly when you call get(). There are different types of exception each of which needs to be handled correctly... – sjlee Jun 21 '13 at 15:26
4

You can use ExecutorCompletionService - it allows you to get Futures in the order they are completed.

You will still have a tiny while loop, and the code you want to invoke after everything is completed will be after the loop. I think that should be ok.

Bozho
  • 588,226
  • 146
  • 1,060
  • 1,140
  • 1
    I don't think that actually helps all that much. An `ExecutorCompletionService` is about picking up the results of individual tasks as they finish. It sounds like @Roman wants to wait for all of the tasks to finish before taking any action. – Tom Anderson Jun 21 '13 at 14:50
  • well, yes, the code that handles that will be invoked after the while(..) loop. There will still be a loop, but a more efficient one – Bozho Jun 21 '13 at 14:52
  • 1
    The simplest while loop would be `List> futures = executorService.invokeAll(tasks); List results = new ArrayList(); for (Future future: futures) results.add(future.get());`. I don't believe the use of an `ExecutorCompletionService` is any more efficient than that. – Tom Anderson Jun 21 '13 at 14:59
  • @TomAnderson You don't need to keep the results in this case, so it can be simpler. `for(Future future: es.invokeAll(tasks)) future.get();` – Peter Lawrey Jun 21 '13 at 15:26
  • 1
    @PeterLawrey - don't forget that `get()` can throw. – parsifal Jun 21 '13 at 15:28
  • @PeterLawrey: Roman mentioned that he wanted to "wait until all tasks are finished, store results, and then" do something else, so we need to collect the results at some point. Might as well do that here. – Tom Anderson Jun 21 '13 at 16:24
  • @TomAnderson I missed the need to store the results. In that case you are correct. – Peter Lawrey Jun 21 '13 at 19:27
0

If you are using JAVA 7, I suggest the Fork/Join framework. I'd explain but this article is much better than anything I can say: http://www.oracle.com/technetwork/articles/java/fork-join-422606.html

MaQy
  • 478
  • 3
  • 10
  • 1
    Fork/join is really only useful for divide-and-conquer algorithms. Its chief benefit is that it ensures that you can combine the results. – parsifal Jun 21 '13 at 15:30
  • Yes, but it might be applicable for this case, it depends on what the tasks do. It was just a suggestion, in case it fits his case. – MaQy Jun 21 '13 at 15:34
0

I would suggest you to use CyclicBarrier. This way you wait till all threads have finished job and then you can start with next set of tasks.

Lokesh
  • 7,810
  • 6
  • 48
  • 78