3

In my grails application I use Spock and Geb to perform functional tests.

Since all test run on the same database, I would like to provide order in which CRUDSpec classes being executed. How this can be specified?

Example

  1. First class tests blog author creation
  2. Second class, assuming first test run successfully, tests post creation
  3. Third class adds comments to the post
snowindy
  • 3,117
  • 9
  • 40
  • 54

1 Answers1

6

It turned out that order can be specified as follows:

grails -Dserver.port=8090 test-app functional: LoginCRUDSpec,PayeeCRUDSpec

Another example using packages from here:

// Run all tests in the “admin” package 
grails test-app functional: admin.**.* 

// Run all tests in the “cart” package 
grails test-app functional: cart.**.*

The ultimate way to order tests with no-arg 'grails test-app' is to name test classes alphabetically.

T001_LoginCRUDSpec
T002_PayeeCRUDSpec
T003_ServiceCRUDSpec
T004_DescrParamCRUDSpec
snowindy
  • 3,117
  • 9
  • 40
  • 54
  • Is it possible to specify order within the spec class (the order in which the tests of a Spec are run)? – Tomas Romero Nov 25 '12 at 16:59
  • I have not seen this yet.. Maybe you should look for this kind of annotation in Spock framework. If you find something, please let me know. – snowindy Nov 25 '12 at 20:23
  • I think it is probably better to setUp and tearDown your specs in such a way that test can be run in any order. I know this is not trivial when dealing with functional tests, but you will benefit from it soon enough. – Hans Westerbeek Nov 26 '12 at 16:59
  • Here is the way to determina order within the spec: 'http://stackoverflow.com/questions/13575972/determine-order-of-execution-of-spock-tests' Basically, using Spock's @Stepwise – Tomas Romero Nov 28 '12 at 03:36
  • Hans, that a valid alternative, because you don't have to warry about data being modified in other tests. Although, in this functional tests I watch them run in a browser, so order is kind of comfortable to have in some cases. Most importantly, I can login as a 'role1' user with some permisions only once for my first 5 tests an then login once as a 'role2' user with other permision for the following tests. But for some specs i think I will follow your alternative. – Tomas Romero Nov 28 '12 at 03:42
  • @Stepwice is applicable only for in-class test-ceses. Not for test-class-file ordering. My app already contains 60 Geb tests. They run for about 3 minutes. If I put them in a single file, to debug a broken test I will have to make N*3min iterations. It would be painful... – snowindy Nov 28 '12 at 07:31
  • I avoid much of the debug pain because I created a test-exec batch-script, which accepts two args: [minTestNumber,maxTestNumber]. Say, I have 4th test-class broken. I run `runTests 0 3` (this saves db-copy after completion of those test files). Then I debug the broken test using `runTests 4` (It uses valid 0-3 phase DB saved copy each time 4 test executes, avoiding broken DB issues). – snowindy Nov 28 '12 at 07:38
  • canotto90, Of cource, I also use @Stepwice for in-file ordering. – snowindy Nov 28 '12 at 07:46