How to test client-side Akka HTTP

Question

I've just started testing out the Akka HTTP Request-Level Client-Side API (Future-Based). One thing I've been struggling to figure out is how to write a unit test for this. Is there a way to mock the response and have the future completed without having to actually do an HTTP request?

I was looking at the API and the testkit package, trying to see how I could use that, only to find in the docs that it actually says:

akka-http-testkit A test harness and set of utilities for verifying server-side service implementations

I was thinking something TestServer (kinda like the TestSource for Akka Streams) and use the server side routing DSL to create the expected response and somehow hook this up the the Http object.

Here is a simplified example of what the function does that I want to test:

object S3Bucket {

  def sampleTextFile(uri: Uri)(
    implicit akkaSystem: ActorSystem,
    akkaMaterializer: ActorMaterializer
  ): Future[String] = {
    val request = Http().singleRequest(HttpRequest(uri = uri))
    request.map { response => Unmarshal(response.entity).to[String] }
  }
}

Answer 1

I think in general terms you've already hit on the fact that the best approach is to mock the response. In Scala, this can be done using Scala Mock http://scalamock.org/

If you arrange your code so that your instance of akka.http.scaladsl.HttpExt is dependency injected into the code which uses it (e.g. as a constructor parameter), then during testing you can inject an instance of mock[HttpExt] rather than one built using the Http apply method.

EDIT: I guess this was voted down for not being specific enough. Here is how I would structure the mocking of your scenario. It is made a little more complicated by all the implicitis.

Code in main:

import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{Uri, HttpResponse, HttpRequest}
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.stream.ActorMaterializer

import scala.concurrent.{ExecutionContext, Future}

trait S3BucketTrait {

  type HttpResponder = HttpRequest => Future[HttpResponse]

  def responder: HttpResponder

  implicit def actorSystem: ActorSystem

  implicit def actorMaterializer: ActorMaterializer

  implicit def ec: ExecutionContext

  def sampleTextFile(uri: Uri): Future[String] = {

    val responseF = responder(HttpRequest(uri = uri))
    responseF.flatMap { response => Unmarshal(response.entity).to[String] }
  }
}

class S3Bucket(implicit val actorSystem: ActorSystem, val actorMaterializer: ActorMaterializer) extends S3BucketTrait {

  override val ec: ExecutionContext = actorSystem.dispatcher

  override def responder = Http().singleRequest(_)
}

Code in test:

import akka.actor.ActorSystem
import akka.http.scaladsl.model._
import akka.stream.ActorMaterializer
import akka.testkit.TestKit
import org.scalatest.{BeforeAndAfterAll, WordSpecLike, Matchers}
import org.scalamock.scalatest.MockFactory
import scala.concurrent._
import scala.concurrent.duration._
import scala.concurrent.Future

class S3BucketSpec extends TestKit(ActorSystem("S3BucketSpec"))
with WordSpecLike with Matchers with MockFactory with BeforeAndAfterAll  {


  class MockS3Bucket(reqRespPairs: Seq[(Uri, String)]) extends S3BucketTrait{

    override implicit val actorSystem = system

    override implicit val ec = actorSystem.dispatcher

    override implicit val actorMaterializer = ActorMaterializer()(system)

    val mock = mockFunction[HttpRequest, Future[HttpResponse]]

    override val responder: HttpResponder = mock

    reqRespPairs.foreach{
      case (uri, respString) =>
        val req = HttpRequest(HttpMethods.GET, uri)
        val resp = HttpResponse(status = StatusCodes.OK, entity = respString)
        mock.expects(req).returning(Future.successful(resp))
    }
  }

  "S3Bucket" should {

    "Marshall responses to Strings" in {
      val mock = new MockS3Bucket(Seq((Uri("http://example.com/1"), "Response 1"), (Uri("http://example.com/2"), "Response 2")))
      Await.result(mock.sampleTextFile("http://example.com/1"), 1 second) should be ("Response 1")
      Await.result(mock.sampleTextFile("http://example.com/2"), 1 second) should be ("Response 2")
    }
  }

  override def afterAll(): Unit = {
    val termination = system.terminate()
    Await.ready(termination, Duration.Inf)
  }
}

build.sbt dependencies:

libraryDependencies += "com.typesafe.akka" % "akka-http-experimental_2.11" % "2.0.1"

libraryDependencies += "org.scalamock" %% "scalamock-scalatest-support" % "3.2" % "test"

libraryDependencies += "org.scalatest" % "scalatest_2.11" % "2.2.6"

libraryDependencies += "com.typesafe.akka" % "akka-testkit_2.11" % "2.4.1"

Answer 2

Considering that you indeed want to write a unit test for your HTTP client you should pretend there is no real server and not cross the network boundary, otherwise you will obviously do integration tests. A long known recipe of enforcing a unit-testable separation in such cases as yours is to split interface and implementation. Just define an interface abstracting access to an external HTTP server and its real and fake implementations as in the following sketch

import akka.actor.Actor
import akka.pattern.pipe
import akka.http.scaladsl.HttpExt
import akka.http.scaladsl.model.{HttpRequest, HttpResponse, StatusCodes}
import scala.concurrent.Future

trait HTTPServer {
  def sendRequest: Future[HttpResponse]
}

class FakeServer extends HTTPServer {
  override def sendRequest: Future[HttpResponse] =
    Future.successful(HttpResponse(StatusCodes.OK))
}

class RealServer extends HTTPServer {

  def http: HttpExt = ??? //can be passed as a constructor parameter for example

  override def sendRequest: Future[HttpResponse] =
    http.singleRequest(HttpRequest(???))
}

class HTTPClientActor(httpServer: HTTPServer) extends Actor {

  override def preStart(): Unit = {
    import context.dispatcher
    httpServer.sendRequest pipeTo self
  }

  override def receive: Receive = ???
}

and test your HTTPClientActor in conjunction with FakeServer.

Answer 3

I was hoping there might be a way to leverage some sort of test actor system but in the absence of that (or some other idiomatic way) I am probably going to do something like this:

object S3Bucket {

  type HttpResponder = HttpRequest => Future[HttpResponse]

  def defaultResponder = Http().singleRequest(_)

  def sampleTextFile(uri: Uri)(
    implicit akkaSystem: ActorSystem,
    akkaMaterializer: ActorMaterializer,
    responder: HttpResponder = defaultResponder
  ): Future[String] = {
    val request = responder(HttpRequest(uri = uri))
    request.map { response => Unmarshal(response.entity).to[String] }
  }
}

Then in my test I can just provide a mock HttpResponder.