How to run specifications sequentially

Question

I want to create few specifications that interoperate with database.

class DocumentSpec extends mutable.Specification with BeforeAfterExample {
  sequential

  def before() = {createDB()}
  def after() = {dropDB()}

  // examples
  // ...
}

Database is created and dropped before and after every example (which is executed sequentially). Everithing works as expected until there is only one spec that works with database. Because specifications are executed parallel, they interfere and fail.

I hope that I'm able to avoid this by instructing specs2 to run tests with side effects sequentially while keeping side effect free tests to run in parallel. Is it possible?

score 45 · Accepted Answer · edited Nov 29 '22 at 17:56

45

I suppose you are using SBT? If so, check the documentation: http://www.scala-sbt.org/release/docs/Detailed-Topics/Parallel-Execution

The relevant SBT setting is parallelExecution. Add this to your project definition:

Test / parallelExecution := false

edited Nov 29 '22 at 17:56

Guillaume Massé

8,004
8
44
57

answered Feb 28 '13 at 21:38

Régis Jean-Gilles

32,541
5
83
97

4

Yes, I'm using SBT. But it doesn't feel natural to specify settings of test suite inside build script. Besides, "parallelExecution in Test := false" ensures that all tests are run sequentially while I need to "to run tests with side effects sequentially while keeping side effect free tests to run in parallel". Maybe using separate db for every db test would be a solution. – Jeriho Mar 01 '13 at 21:32
2

Or simpler, put thoses tests in a distinct project and redefine `parallelExecution` for this project only. – Régis Jean-Gilles Mar 01 '13 at 23:58

Ivan Mushketyk · Answer 2 · 2018-03-30T13:25:29.717

42

If you want to run single Specification in specs2 sequentially just add sequential method call at the beginning of your Specification. For example:

class MyTest extends Specification {
  // Set sequential execution
  sequential

  // This tests will be executed sequentially
  "my test" should {
    "add numbers" in {
      (1 + 1) should be equalTo 2
    }

    "multiply numbers" in {
      (2 * 2) should be equalTo 4
    }
  }
}

UPDATE: As @jsears correctly mentioned in the comments, this will make tests to run sequentially in a single spec! Other specs may still run in parallel.

edited Mar 30 '18 at 13:25

answered Apr 19 '14 at 13:56

Ivan Mushketyk

8,107
7
50
67

7

You know, at my workplace, we're *still* not sure that this actually runs the tests in sequential order, or if it even prevents them running in parallel at all. – 2rs2ts Mar 13 '15 at 00:11
1

Weird... It worked for me and for many other people. If you can come up with an example when you tests are not executed sequentially with this method call, you should probably report a bug to spec2. – Ivan Mushketyk Mar 14 '15 at 00:31
It does. See the response of Eric, the creator of Specs2 , https://stackoverflow.com/questions/8026866/parallel-execution-of-tests – Raymond Chenon Feb 19 '18 at 10:27
1

"sequential" runs the examples in a single spec in serial. Other specs may still run in parallel, so if you have other specs with a shared fixture, you may see that happen in parallel. – jsears Mar 28 '18 at 19:05
@jsears That's correct. I'll update the answer to reflect this. – Ivan Mushketyk Mar 30 '18 at 13:24

score 7 · Answer 3 · answered Nov 04 '16 at 11:35

Meanwhile there is a better solution (http://www.scala-sbt.org/release/docs/Parallel-Execution.html):

sbt 0.12.0 introduces a general infrastructure for restricting task concurrency beyond the usual ordering declarations.

This configuration will run all tests sequential, also if they are in subprojects:

concurrentRestrictions in Global := Seq(
  Tags.limit(Tags.CPU, 2),
  Tags.limit(Tags.Network, 10),
  Tags.limit(Tags.Test, 1),
  Tags.limitAll( 15 )
)

I haven't tested if this can be overridden by each sub-project, so the sub-project can run its tests in parallel.

How to run specifications sequentially

3 Answers3

Linked