3

I am trying to run 3 operations in parallel using the CompletableFuture approach. Now these 3 operations return different types so need to retrieve the data separately. Here is what i am trying to do:

CompletableFuture<List<A>> aFuture = CompletableFuture.supplyAsync (() -> getAList());
CompletableFuture<Map<String,B> bFuture = CompletableFuture.supplyAsync (() -> getBMap());
CompletableFuture<Map<String,C> cFuture = CompletableFuture.supplyAsync (() -> getCMap());

CompletableFuture<Void> combinedFuture =
                    CompletableFuture.allOf (aFuture, bFuture, cFuture);

combinedFuture.get(); (or join())

List<A> aData = aFuture.get(); (or join)
Map<String, C> bData = bFuture.get(); (or join)
Map<String, C> cData = cFuture.get(); (or join)

This does the job and works but i am trying to understand if we need to do these gets/joins on combined future as well as individual ones and if there is a better way to do this.

Also i tried using then whenComplete() approach but then the variables i want to assign the returned data are inside the method so i am getting a "The final local variable cannot be assigned, since it is defined in an enclosing type in Java" error and i don't want to move them to the class level.

looking for some expert/alternate opinions. Thank you in advance

SG

sg1973
  • 91
  • 6
  • If you really are retrieving the results of all three CompletableFutures in the same place, one after another, then there is no need for CompletableFuture.allOf. Calling all three `get()` methods effectively will wait for them all to complete anyway. – VGR Apr 29 '22 at 12:14
  • Thanks, How else to do it if we are using ofAll to get the results? I can use the thenApply option because of variable scope mentioned above – sg1973 Apr 30 '22 at 01:54
  • Just remove the creation and use of `combinedFuture`. It is not needed. Simply calling `get()` on each of the individual futures is sufficient. Waiting for them one at a time will guarantee all of them have finished. – VGR Apr 30 '22 at 13:11
  • Thanks. Yes but if i want to use the allOf, how would i retrieve the data from the futures in that case? – sg1973 Apr 30 '22 at 13:31
  • The only way to get individual futures’ return values is to keep references to the individual futures, so you can call their get() methods. The object returned by allOf does not provide any way to access the individual constituent futures. – VGR Apr 30 '22 at 13:56
  • Thanks. Yes thats why i have no choice but to do the gets on them. The other alternative is to do whenComplete to make it concise but that will do the same get/joins on the individual futures and also can’t use method local variables. – sg1973 May 01 '22 at 00:43
  • Is there a reason you don’t want to call `get()` for each future? – VGR May 01 '22 at 00:56
  • Are you suggesting to do just that instead of calling allOf? The reason is when we do that the futures are not going to be invoked until the get() is called no? If i understand correctly, allOf() call helps grouping them which we can then use the get/join to get the individual results upon completion. Do you see it done any other way? – sg1973 May 01 '22 at 17:28
  • Are you suggesting we don't need allOf call and instead just do get/join on the Futures? – sg1973 May 01 '22 at 17:37
  • Yes, I’m saying that calling each future’s get(), one at a time, will effectively do the same thing as waiting for the result of allOf. In both cases, every future will have completed. – VGR May 01 '22 at 18:22
  • VGR already addressed, but there are a few caveats. If you want ALL the futures to be completed before you do ANYTHING, then you need to call `allOf(......).get()`, then you must call `get()` for each of the 3 futures. However, if you merely want each future to be completed before it is processed (while also allowing the other, unreached futures to continue in the background), then you just do what @VGR said. – davidalayachew May 01 '22 at 23:21
  • You can use the "hidden Applicative" to run multiple futures in parallel and collect individual results when all completed. See https://stackoverflow.com/a/70724491/402428 – michid May 03 '22 at 09:06

1 Answers1

1

Calling get or join just implies “wait for the completion”. It has no influence of the completion itself.

When you call CompletableFuture.supplyAsync(() -> getAList()), this method will submit the evaluation of the supplier to the common pool immediately. The caller’s only influence on the execution of getAList() is the fact that the JVM will terminate if there are no non-daemon threads running. This is a common error in simple test programs, incorporating a main method that doesn’t wait for completion. Otherwise, the execution of getAList() will complete, regardless of whether its result will ever be queried.

So when you use

CompletableFuture<List<A>> aFuture = CompletableFuture.supplyAsync(() -> getAList());
CompletableFuture<Map<String,B>> bFuture=CompletableFuture.supplyAsync(() -> getBMap());
CompletableFuture<Map<String,C>> cFuture=CompletableFuture.supplyAsync(() -> getCMap());

List<A> aData = aFuture.join();
Map<String, B> bData = bFuture.join();
Map<String, C> cData = cFuture.join();

The three subsequent supplyAsync calls ensure that the three operations might run concurrently. The three join() calls only wait for the result and when the third join() returned, you know that all three operations are completed. It’s possible that the first join() returns at a time when aFuture has been completed, but either or both of the other operations are still running, but that doesn’t matter for three independent operations.

When you execute CompletableFuture.allOf(aFuture, bFuture, cFuture).join(); before the individual join() calls, it ensures that all three operations completed before the first individual join() call, but as said, it has no impact when all three operations are independent and you’re not relying on some side effect of their execution (which you shouldn’t in general).

The actual purpose of allOf is to construct a new future when you do not want to wait for the result immediately. E.g.

record Result(List<A> aData, Map<String, B> bData, Map<String, C> cData) {}

CompletableFuture<Result> r = CompletableFuture.allOf(aFuture, bFuture, cFuture)
    .thenApply(v -> new Result(aFuture.join(), bFuture.join(), cFuture.join()));
// return r or pass it to some other code...

here, the use of allOf is preferable to, e.g.

CompletableFuture<Result> r = CompletableFuture.supplyAsync(
    () -> new Result(aFuture.join(), bFuture.join(), cFuture.join()));

because the latter might block a worker thread when join() is called from the supplier. The underlying framework might compensate when it detects this, e.g. start a new thread, but this is still an expensive operation. In contrast, the function chained to allOf is only evaluated after all futures completed, so all embedded join() calls are guaranteed to return immediately.

For a small number of futures, there’s still an alternative to allOf, e.g.

var r = aFuture.thenCompose(a ->
            bFuture.thenCombine(cFuture, (b, c) -> new Result(a, b, c)));
Holger
  • 285,553
  • 42
  • 434
  • 765