81

How large is Scala's thread pool for futures?

My Scala application makes many millions of future {}s and I wonder if there is anything I can do to optimize them by configuring a thread pool.

Thank you.

Erik Kaplun
  • 37,128
  • 15
  • 99
  • 111
  • Slick 3.0 uses own connection and threadpool so why do we need to provide implicit executioncontext to slick when it manages own thread pool – Rahul Gulabani Mar 25 '17 at 10:06
  • 1
    @RahulGulabani, from "essential slick" book : `The reason is that map, flatMap methods of Action allows you to call arbitrary code when joining the actions together. Slick cannot allow that code to be run on its own execution context, because it has no way to know if you are going to tie up Slicks threads for a long time.` – srzhio Jan 14 '20 at 13:49

4 Answers4

159

This answer is from monkjack, a comment from the accepted answer. However, one can miss this great answer so I'm reposting it here.

implicit val ec = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10))

If you just need to change the thread pool count, just use the global executor and pass the following system properties.

-Dscala.concurrent.context.numThreads=8 -Dscala.concurrent.context.maxThreads=8
Bienvenido David
  • 4,118
  • 1
  • 25
  • 16
  • 1
    I tried both of these with the value as 5, and I'm still seeing up to 8 threads running concurrently. – micseydel Apr 25 '19 at 17:50
90

You can specify your own ExecutionContext that your futures will run in, instead of importing the global implicit ExecutionContext.

import java.util.concurrent.Executors
import scala.concurrent._

implicit val ec = new ExecutionContext {
    val threadPool = Executors.newFixedThreadPool(1000)

    def execute(runnable: Runnable) {
        threadPool.submit(runnable)
    }

    def reportFailure(t: Throwable) {}
}
Jämes
  • 6,945
  • 4
  • 40
  • 56
Josh Gao
  • 2,525
  • 23
  • 21
  • 74
    Great answer, you can reduce the boilerplate a bit by using the helper methods on ExecutionContext that let you instantiate directly from a given Executor. Eg implicit val ec = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(10)) – sksamuel Apr 28 '13 at 20:02
  • 1
    Granted this all is nice, but is there a real limit in threads on implicits.global ? If so is this configurable like akka via application.conf? – Nick Apr 24 '14 at 01:03
  • 6
    @Nick yes, implicits.global is only 1 thread per CPU core. Optimal for cpu-bound tasks. But for classic blocking IO (eg jdbc) it's a performance disaster. – Ben Hutchison Nov 04 '14 at 01:35
  • 2
    I needed to add a call to shut down the thread pool after using this, or the program never terminates ... def shutdown() = threadPool.shutdown() – justinhj May 03 '15 at 19:03
  • 2
    Why does it shutdown in the "normal" case, but not when we set the implicit to something else? – hbogert Aug 31 '15 at 10:28
3

best way to specify threadpool in scala futures:

implicit val ec = new ExecutionContext {
      val threadPool = Executors.newFixedThreadPool(conf.getInt("5"));
      override def reportFailure(cause: Throwable): Unit = {};
      override def execute(runnable: Runnable): Unit = threadPool.submit(runnable);
      def shutdown() = threadPool.shutdown();
    }
Renzo
  • 26,848
  • 5
  • 49
  • 61
S'chn T'gai Spock
  • 1,203
  • 18
  • 16
-1
class ThreadPoolExecutionContext(val executionContext: ExecutionContext)

object ThreadPoolExecutionContext {

  val executionContextProvider: ThreadPoolExecutionContext = {
    try {
      val executionContextExecutor: ExecutionContextExecutor = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(25))
      new ThreadPoolExecutionContext(executionContextExecutor)
    } catch {
      case exception: Exception => {
        Log.error("Failed to create thread pool", exception)
        throw exception
      }
    }
  }
}
VAIBHAV GOUR
  • 167
  • 1
  • 3