1

I have a WDIO project that has many tests. Some tests need to be run consecutively while other tests can run in parallel.

I cannot run all tests in parallel because the tests that need to be run consecutively will fail, and I cannot run all tests consecutively because it would take far too long for the execution to finish.

For these reasons I need to find a way to run these tests both consecutively and in parallel. Is it possible to configure this WDIO project to accomplish this?

I run these tests through SauceLabs and understand that I can set the maxInstances variable to as many VMs as I'd like to run in parallel. Is it possible to set certain tests to use a high maxInstance while other tests have a maxInstance of 1?

Or perhaps there is a way to use logic logic via the test directories to run certain tests in parallel and others consecutively?

For example, if I have these tests:

'./tests/parallel/one.js',
'./tests/parallel/two.js',
'./tests/consecutive/three.js',
'./tests/consecutive/four.js',

Could I create some logic such as:

if(spec.includes('/consecutive/'){
    //Do not run until other '/consecutive/' test finishes execution
} else {
    //Run in parallel
}

How can I configure this WDIO project to run tests both consecutively and in parallel? Thank you!

Brian
  • 135
  • 4
  • 16

1 Answers1

1

You could create 2 separate conf.js files.

//concurrent.conf.js
exports.config = {
    // ==================
    // Specify Test Files
    // ==================
    specs: [
        './test/concurrent/**/*.js'
    ],
    maxInstances: 1,

and have one for parallel. To reduce duplication, create a shared conf.js and then simply override the appropriate values.

//parallel.conf.js
const {config} = require('./shared.conf');

config.specs = [
        './test/parallel/**/*.js'
    ],
config.maxInstances = 100,

And then when you run your tests you can do:

//parallel
wdio test/configs/parallel.conf.js
//concurrent
wdio test/configs/concurrent.conf.js

Here's an example of how to have a shared config file. And other config files using the shared one

Nikolay Advolodkin
  • 1,820
  • 2
  • 24
  • 28
  • Thank you so much!! This definitely seems like the path I need to take. However, there is one final issue: Both `conf` files must run in the same process. I've tried two possible solutions: `"test": "wdio parallel.conf.js --suite test && wdio consecutive.conf.js --suite test"` This runs both `conf` files, but one after the other. `"test": "wdio parallel.conf.js --suite test & wdio consecutive.conf.js --suite test"` This runs both `conf` files at the same time, but in two separate processes. Is it possible to have both files run in the same process at the same time? Thank you again!! – Brian Aug 18 '21 at 21:41
  • @Brian why must they run in the same process? – Nikolay Advolodkin Aug 19 '21 at 15:23
  • The project utilizes a custom feature which generates a report at the end of execution. Having the two `conf` files execute in parallel breaks the reporting system completely due to two processes being created. Having the two `conf` files run in a single process would resolve this issue. Do you think this is possible? – Brian Aug 19 '21 at 17:25
  • @Brian hmmm, my solution wouldn't work then if you're trying to get your reporting right. I think you have a few options. First, use the solution above and not worry so much about reporting output. Second, try to find a new solution that meets your needs, maybe possible, I'm not sure how. The former seems like the better option as it's weird to me to structure test code to fit reporting needs. Console and my CI systems typically provide me all the logging that I need. – Nikolay Advolodkin Aug 19 '21 at 18:21
  • Thank you for you help! I'll see if I can make it work – Brian Aug 19 '21 at 19:17