How to use Akka-TestKit TestProbe correctly?

Question

I expanded the example from http://doc.akka.io/docs/akka/snapshot/scala/testing.html#Using_Multiple_Probe_Actors.

import akka.actor._
import akka.testkit.{TestProbe, TestKit}
import org.scalatest.{Suites, BeforeAndAfter, BeforeAndAfterAll, FlatSpecLike}
import scala.concurrent.duration._

class TestProbesTestSuites extends Suites(new TestProbesTest)

class TestProbesTest extends TestKit(ActorSystem("TestProbesTestSystem")) with FlatSpecLike with BeforeAndAfterAll with BeforeAndAfter {
  override def afterAll: Unit = {
    TestKit.shutdownActorSystem(system)
  }

  "A TestProbeTest" should "test TestProbes" in {
    val actorRef = system.actorOf(Props[TestActor], "TestActor")
    val tester1 = TestProbe()
    val tester2 = TestProbe()

    Thread.sleep(500.milliseconds.toMillis)

    actorRef ! (tester1.ref, tester2.ref)
    // When you comment the next line the test fails
    tester1.expectMsg(500.milliseconds, "Hello")
    tester2.expectMsg(500.milliseconds, "Hello")

    // Alternative test
//    Thread.sleep(500.milliseconds.toMillis)
//    tester1.expectMsg(0.milliseconds, "Hello")
//    tester2.expectMsg(0.milliseconds, "Hello")
    ()
  }
}

class TestActor extends Actor with ActorLogging {
  override def receive: Receive = {
    case (actorRef1: ActorRef, actorRef2: ActorRef) => {
      // When you change the schedule time in the next line to 100.milliseconds the test fails
      context.system.scheduler.scheduleOnce(400.milliseconds, actorRef1,  "Hello")(context.system.dispatcher)
      context.system.scheduler.scheduleOnce(800.milliseconds, actorRef2,  "Hello")(context.system.dispatcher)
    }
    case x => log.warning(x.toString)
  }
}

I do not think that this a correct or good usage for a test. When I remove the tester1.expectMsg(500.milliseconds, "Hello") the test fails, so testing of tester2 dependes on testing tester1. In my opinion this bad.

Also changing line context.system.scheduler.scheduleOnce(400.milliseconds, actorRef1, "Hello")(context.system.dispatcher) to a delay of 100 milliseconds let the test fail. So testing message 2 depends on sending message 1. In my opinion this is bad, too.

To solve this I would add a Thread.sleep after sending the message and change the wait time for both #expectMsg to 0 milliseconds. Thread.sleep does also not look good for me in tests, but I think it is a must in actor testing. Is this the right way?

I thought TestProbe is made for testing multiple actors. But for me the wait time parameter of #expectMsg is quite useless, when testing multiple actors.

Any remarks are welcome.

If you are scheduling the message to `tester2` to be received in 800 milliseconds then it's always going to fail if you expect it to be received in 500 milliseconds. It just happens to work because the first assertion causes a wait of 400ms beforehand. — cmbaxter, Aug 04 '15 at 13:25

score 3 · Answer 1 · answered Aug 04 '15 at 12:51

3

The test probe works synchronously. The expectMsg method is blocking. And it's good, because you want to run tests in one thread and have control over it. There are no race conditions this way.

The timeout while expecting messages is measured from the time the line is read, which means that the 500 millis in tester2.expectMsg(500.milliseconds, "Hello") will start counting down only after tester1 receives his message.

If tester1 doesn't get the message, you want to fail the test immediately, you don't need to continue to tester2.

On the other hand, scheduler is done asynchronously, in multiple threads, and the countdown starts right away because there is no blocking. So the scheduled 400 millis and 800 millis will start counting down (nearly) at the same time.

So changing the 400 millis to 100 millis means that the second message will still be sent approximately 800 millis from start, but tester1 will be expecting it 100 + 500 = 600 millis from start. That's why it fails.

answered Aug 04 '15 at 12:51

Quizzie

879
4
15

Thanks for your post @Quizzie. I understand WHY the test fails, but I do not think it is good. I am quite new in testing, but as I assume unit tests should be single threaded, integration tests mulit threaded. I can not avoid any race conditions when using actors (except TestActorRef), I guess. TestProbe should be suitable for integration tests. How would you implement two TestProbes, which should receive a message within 500ms. I thought the example on my link does it (and I think most of the readers would think so too), but probe1 has 500ms and probe2 500-1000ms. – hami Aug 04 '15 at 13:04
I would just make the first timeout 500 millis and the second timeout 0 millis. Or you can have only one probe and pass it as both arguments and expect both messages with one method (expectMsgAllOf). Single threaded, synchronous testing is the way to go even in integration tests. – Quizzie Aug 04 '15 at 13:31
I definately need different actors for testing, so #expectMsgAllOf is not a solution. Second timeout to 0ms is a soultion. Can I also integrate #expectNoMsg? My target would be to have several probes, f.e. within the next 500ms probe1 and 2 should receive message x and probes 3 and 4 should receive no message. – hami Aug 04 '15 at 14:58

score 3 · Accepted Answer · answered Aug 04 '15 at 13:51

3

You could try a within block instead like so:

within(800.milliseconds, 900.milliseconds){
  tester1.expectMsg("Hello")
  tester2.expectMsg("Hello")
}

That way if you comment out the assertion to tester1 the test still passes.

answered Aug 04 '15 at 13:51

cmbaxter

35,283
4
86
95

Nice, your code is working. But I do not really understand it. http://doc.akka.io/docs/akka/snapshot/scala/testing.html#Caution_about_Timing_Assertions says the deadline is local to each probe. So I would expect each probe should finish within 800 to 900ms. That is not the case. For me it seems that the whole block has to finish within 800 to 900ms. Right? But then I do not understand the documentation. Can I also use this for #expectNoMsg(). My target would be to have several probes, f.e. within the next 500ms probe1 and 2 should receive message x and probes 3 and 4 should receive no message. – hami Aug 04 '15 at 14:51
Within means that the entirety of that block (from `{` to `}`) must finish within the min and max defined (800ms and 900ms here). You can use this in lieu of individual assertion specific timeouts when you are more relaxed with your assertions and only care that a bunch of stuff has happened within a certain timeframe. Yes, you should be able to use it with `expectNoMsg()` too. – cmbaxter Aug 04 '15 at 14:55
Sorry to be so fussy, but I really want to understand it and the docu is not clear enough for me. Are the calls still blocking? When I would change tester1.expectMsg("Hello") change to tester1.expecNotMsg(), would tester1 block until 900ms and then check tester2 or are expect functions in a within block somehow parallel? – hami Aug 04 '15 at 15:08
I don't know what is missing here. But according to the docs, Test Probe in the within block will still use default Timeout. I tried with within clause and it still timed out in 3 seconds. Instead I made use of tester.expectMsg(duration, message), and it worked for me. – Robin Loxley Sep 29 '16 at 07:20

How to use Akka-TestKit TestProbe correctly?

2 Answers2