Best practice for Slick 2.1 / 3 execution context usage

Question

We use Slick (2.1.0) with Spray-io (1.3.3). Currently we are facing an issue because we use the same execution context for both the Spray HTTP API part and background running jobs accessing the same database. All database / blocking calls are wrapped in futures using the same scala.concurrent.ExecutionContext.global execution context.

When the background jobs start doing heavy work, they'll consume all available threads, which will lead to timeouts on the API side since their aren't any available threads to handle the API work. The obvious solution would be to use different execution contexts for both parts with a total thread count not higher than the configured DB connection pool (HikariCP). (as partially suggested here https://www.playframework.com/documentation/2.1.0/ThreadPools#Many-specific-thread-pools) but how would such a setup work with Slick 3 where the execution context is tied to the DB configuration itself?

score 3 · Accepted Answer · answered Feb 21 '16 at 14:31

Slick3 comes with own execution context and number of threads are configurable.You can tweak all the connection pool setting for example(MySQL):

dev-dbconf={
dataSourceClass  =  "com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
numThreads       =  10 //for execution context 
maxConnections   =  10
minConnections   =  5
connectionTimeout = 10000
initializationFailFast = false
properties {
    user         = "root"
    password     = "root"
    databaseName = "db_name"
    serverName   = "localhost"
}

}

In this config you can change number of thread according your requirement. I would like advise you never used "scala.concurrent.ExecutionContext.global" for IO. Because default ExecutionContext comes with fork-join thread pool which is not good for IO.You can create own thread pool for IO:

import scala.concurrent.ExecutionContext
import java.util.concurrent.Executors

object MyExecutionContext {

    private val concorrency = Runtime.getRuntime.availableProcessors()
    private val factor =  3 // get from configuration  file
    private val noOfThread = concorrency * factor
    implicit val ioThreadPool: ExecutionContext = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(noOfThread))

}
  // Use this execution context for IO instead of scala execution context.
 import MyExecutionContext.ioThreadPool

 Future{
 // your blocking IO code
 }

You can change noOfThread according to your requirement. It would be good if you set number of thread according number processors in your machine.

For more info, your can see Best Practices for Using Slick on Production and Slick Doc.

Using a fork-join thread pool is indeed not a good idea but how can I make sure that the HTTP API DB calls are always prioritised over the background jobs using slick 3.0 since the same execution context is used? — Chris, Feb 22 '16 at 06:49
I am not able to understand exactly your problem. For that you need show me some code. If you want to use scala default ExecutionContext , It is ok. — Sky, Feb 22 '16 at 15:27
You need to increase noOfThread in default threadpool. By default it is x1.means noOfThread =(number if processor) * 1. you can set by 1) scala.concurrent.context.minThreads 2)scala.concurrent.context.numThreads 3)scala.concurrent.context.maxThreads For more detail see : http://stackoverflow.com/questions/14207762/set-the-parallelism-level-for-all-collections-in-scala-2-10/14291710#14291710 and https://github.com/scala/scala/blob/2.11.x/src/library/scala/concurrent/impl/ExecutionContextImpl.scala — Sky, Feb 22 '16 at 15:33

Best practice for Slick 2.1 / 3 execution context usage

1 Answers1