1

I am using Scalatest, FlatSpec, Spring, Selenium DSL and BeforeAndAfterAll. One of these things seems to stop ParallelTestExecution working properly. This is what happens when I run a class with two tests:

  1. One Browser opens and does some beforeAll stuff (but not Spring stuff)
  2. Another browser opens and does the beforeAll stuff
  3. Second browser gets used for first test then closes
  4. Another browser opens and does beforeAll stuff followed by second test
  5. First and Third browsers close

So basically the test runs exactly the same as without ParallelTestExecution except that an extra window has opened?

shmish111
  • 3,697
  • 5
  • 30
  • 52

1 Answers1

11

I think you may be observing two different effects. First, ParallelTestExecution extends OneInstancePerTest. It runs each test in its own instance to reduce the likelihood of shared mutable state between tests introducing concurrency Heisenbugs into your tests. But the way it does this is the initial instance of your test class creates an instance for each test and passes those to the distributor (if defined), which will run them in parallel. So since you have two tests, you will get three instances of your test class--the initial instance that runs on the main thread, and the two test-specific instances, one per test, that can run those tests in parallel. Since your beforeAll and afterAll methods have a side effect of creating and closing a web browser, you see that side effect three times.

The other thing that may be happening is that ParallelTestExecution will only run tests in parallel if you tell ScalaTest you want parallel execution in general. If you're using Runner, that's done by passing in -P. Otherwise the distributor will not be defined, in which case ParallelTestExecution just executes the tests sequentially on the main thread (the thread that called run).

ParallelTestExecution is intended for rather rare cases when you actually need to execute tests in the same test class in parallel. An example might be a test class that has lots of very slow tests. In most cases I expect ScalaTest's default approach of running Suites in parallel should give you as good a performance boost as running tests in parallel. To get that kind of parallel execution you need not mix in any trait (i.e., no need for ParallelTestExecution). Just pass -P to Runner, or tell sbt to run ScalaTest in parallel, etc.

Also, BeforeAndAfterAll is meant for things that need to happen before and after all tests and nested suites. If you want each test to have its own browser, then you probably want to use BeforeAndAfterEach instead. This would give you just two browsers popping up, not three, in the ParallelTestExecution case. If you really wanted all tests to share the same browser, then I'd check the Selenium documentation to ensure that's possible. It may be that Selenium only let's you do one interaction at a time with a given WebBrowser driver.

In summary, if I am guessing correctly what you're really trying to accomplish, I'd drop ParallelTestExecution, change BeforeAndAfterAll to BeforeAndAfterEach (or use withFixture), and pass -P to Runner (directly, via ant or Maven) or ask sbt to run ScalaTest in parallel.

Bill Venners
  • 3,549
  • 20
  • 15
  • Great answer. My tests all test one feature and so IMO belong in one class. Unfortunately each test does different things that take some time so it would be advantageous if I could get them running in parallel. When I wrote the test I didn't think about ParallelTestExecution so that's why I used BeforeAndAfterAll. I see now that if I want to use ParallelTestExecution I should use BeforeAndAfterEach. I'll give it a go on Tuesday. Do you know by chance how I enable parallel execution using the scalatest maven plugin and also Intellij? – shmish111 Apr 01 '13 at 00:38
  • This worked perfectly thanks. Needed to add -P test parameter in IntelliJ and true in maven scalatest plugin. Also needed to add an afterAll method to quit the distributor instance of webdriver. – shmish111 Apr 02 '13 at 09:17