Spray.io: When (not) to use non-blocking route handling?

Question

If we are thinking of production grade REST API, should we use non-blocking as much as possible, e.g.

def insertDbAsync(rows: RowList): Future[Unit] = ...
...
val route =
path("database" / "insertRowList") {
  post {
    entity(as[RowList]) { rows =>
      log.info(s"${rows.length} rows received")
      val async = insertDbAsync(rows)
      onComplete(async) {
        case Success(response) =>
          complete("success")
        case Failure(t) =>
          complete("error")
      }
    }
  }
}

I'm thinking that the answer will most likely be a 'yes', but what are some guidelines in deciding what should and should not be a blocking code, and why?

It is always good to have non-blocking code in any of the applications. But which part of code would be non-blocking that you have to decide at the time of designing your app. Sometimes you need result to process other steps and that time it is always good to use blocking code and if your code dosen't required result data wrappe your result with Future and make non-blocking code. — Nishan, Apr 01 '15 at 08:05

dk14 · Accepted Answer · 2015-07-08T03:05:32.917

Spray uses Akka as underlying platform, so recommendations are same as for actors (Blocking Needs Careful Management). Blocking code may require too much threads, which may:

kill actor's lightweightness: millions of actors may operate on one thread by default. Let's say one non-blocked actor requires 0.001 threads for example. One blocked actor (which blocking time is, let's say, 100 times more than usual) will take 1 thread avg (not always same thread). First, The more threads you have - the more memory you loose - every blocked thread holds full callstack allocated before blocking, including references from stack (so GC can't erase them). Second, if you have more than number_of_processors threads - you will loose the performance. Third, if you use some dynamical pool - adding new thread may take some significant amount of time.
cause thread's starvation - you may have pool filled with threads, which doing nothing - so new tasks can't be processed before blocking operation complete (0 % CPU load, but 100500 messages waiting to be processed). It may even cause deadlocks. However, Akka uses Fork-Join-Pool by default so if your blocking code is managed (surrounded with scala.concurrent.blocking - Await.result have such surrounding inside ) - it will prevent starvation by cost of creating new thread instead of blocked one, but it won't compensate other problems.
traditionally cause deadlocks, so it's bad for design

If code is blocking from outside, you can surround it with future:

 import scala.concurrent._
 val f = Future {
     someNonBlockingCode()
     blocking { //mark this thread as "blocked" so fork-join-pool may create another one to compensate
        someBlocking()
     }  
 }

Inside separate actor:

 f pipeTo sender //will send the result to `sender` actor

Inside spray routing:

 onComplete(f) { .. }

It's better to execute such futures inside separate pool/dispatcher (fork-join-pool based).

P.S. As an alternative to futures (they may not be much convinient from design perspectvive) you may consider Akka I/O, Continuations/Coroutines, Actor's pools (also inside separate dispatcher), Disruptor etc.

This is a good advice. Do you think that `Future`-ing every single IO-related call (e.g. reading/writing to file / database / network) is a good practice? Is there any situation where it might be a bad idea? — lolski, Apr 01 '15 at 13:25
Yes it's a good practice. it's a bad idea if an api provides asynchronous listener already. Or, obviously, if you use some shared state (concurrent collections or STM may help here) or even free variables (as they will lock some memory from JVM's GC). Futures are more lightweight than actors, so no much overhead. In general case, it may be hard to manage futures from design perspective, so you may consider something like reactive streams (akka streams for instance) or continuations/[coroutines](http://jim-mcbeath.blogspot.com/2010/09/scala-coroutines.html). — dk14, Apr 01 '15 at 14:00
More info: [Blocking Needs Careful Management](http://doc.akka.io/docs/akka/snapshot/general/actor-systems.html). For example you may achieve a better throughput if you create a [pool](http://doc.akka.io/docs/akka/snapshot/scala/routing.html) (not thread-pool - actors pool) with number of actors equals maximum number of connections to DB (and with separate dispatcher of course). — dk14, Apr 01 '15 at 14:11

score 0 · Answer 2 · answered Apr 01 '15 at 09:26

0

If you're using spray everything must be non-blocking as a matter of correctness - otherwise you'll block the (very small number of) dispatch threads and your server will stop responding.

answered Apr 01 '15 at 09:26

lmm

17,386
3
26
37

Spray.io: When (not) to use non-blocking route handling?

2 Answers2

Linked