26

I have a Project Reactor chain which includes a blocking task (a network call, we need to wait for response). I'd like to run multiple blocking tasks concurrently.

It seems like either ParallelFlux or flatMap() could be used, bare-bone examples:

Flux.just(1)
    .repeat(10)
    .parallel(3)
    .runOn(Schedulers.elastic())
    .doOnNext(i -> blockingTask())
    .sequential()
    .subscribe()

or

Flux.just(1)
    .repeat(10)
    .flatMap(i -> Mono.fromCallable(() -> {blockingTask(); return i;}).subscribeOn(Schedulers.elastic()), 3)
    .subscribe();

What are the merits of the two techniques? Is one to be preferred over the other? Are there any alternatives?

Corin Fletcher
  • 1,611
  • 1
  • 17
  • 25

1 Answers1

28

parallel is tailored for parallelization of tasks for performance purposes, and dispatching of work between "rails" or "groups", each of which get their own execution context from the Scheduler you pass to runOn. In short, it will put all your CPU cores to work if you do CPU intensive work. But you're doing I/O bound work...

So in your case, flatMap is a better candidate. That use of flatMap for parallelization is more about orchestration.

These are pretty much the 2 alternatives, if you don't count the slightly different flavor of flatMap that flatMapSequential is (concatMap doesn't really allow for parallelization).

Simon Baslé
  • 27,105
  • 5
  • 69
  • 70
  • 13
    I don't really see the difference. With flatMap you still have to subscribeOn different "threads" – IceMan May 07 '18 at 20:44
  • 1
    Yep, I agree with @IceMan. How does flatMap provide parallelization without having it to subscribeOn(Scheduler)? – Vivek Sethi Dec 23 '19 at 15:17
  • I believe you overlooked `.subscribeOn(Schedulers.elastic()), 3)` - indeed, in fact it is there, in the example given, on the same line where `flatMap` is. And I agree, it is easy to overlook as all other operators are on separate lines. – barthand Jan 30 '21 at 23:35
  • 3
    agreed, with a _blocking_ example the difference is hard to see. but if the source(s) that `flatMap` work with are non-blocking (I/O being a prime candidate for conversion to non-blocking implementation), then `flatMap` can truly shine there. the operator will act as an event loop, getting notification from the IO publisher whenever it is ready, and ensuring all these elements are published to the downstream serially – Simon Baslé Feb 02 '21 at 13:47