1

We are setuping our current solution on azure devops.

We have a big bunch of tests(some unit tests, some integrations tests) all running with NUnit.

We have configured the test like this:

- task: VSTest@2
    timeoutInMinutes: 600
    inputs:
      testSelector: 'testAssemblies'
      testAssemblyVer2: '.Test.dll'
      searchFolder: '$(System.DefaultWorkingDirectory)/$(buildConfiguration)'
      codeCoverageEnabled: false
      platform: 'Any CPU'
      configuration: '$(buildConfiguration)'
      rerunFailedTests: false
      pathtoCustomTestAdapters: 'Solution/packages/NUnit3TestAdapter.3.12.0/build/net35'
      minimumExpectedTests: 1000
      runInParallel: true
      distributionBatchType: basedOnAssembly
      failOnMinTestsNotRun: true
      resultsFolder: 'testResults'

It's working, but it's taking ages(I'm speaking of 5+ hours), and I'm searching for way to speed things up.

One weird thing: Even with the runInParallel: true, there is only one process(testhost) running, and consuming like 3-6% of the CPU.

What should I do to hurt my system CPU/hard disk/... and improve my test speeds?

J4N
  • 19,480
  • 39
  • 187
  • 340
  • Do you run these test on self hosted agent? – Krzysztof Madej Oct 01 '20 at 19:27
  • `runInParallerl` override MaxCpuCount what does `typically, a value of 0 indicates that up to all of the available free cores may be used.` – Krzysztof Madej Oct 01 '20 at 19:33
  • How many test assemblies are you running? – Charlie Oct 01 '20 at 23:00
  • @KrzysztofMadej yes, I forgot to mention that, it's on a self hosted agent. – J4N Oct 02 '20 at 03:30
  • @Charlie around 20,why? – J4N Oct 02 '20 at 03:30
  • How many cores does your proc have? – Krzysztof Madej Oct 02 '20 at 03:56
  • @KrzysztofMadej it depends, we have several agents, one has 6 core and 12 logical processor. Another one has 4 core and 4 logical processor – J4N Oct 02 '20 at 06:15
  • Not all test code that has been written can be performed in a parallel and safe manner, so you could try to check whether your assembly meets the conditions and How about the result when you execute this test in your local in a parallel, check if it work.https://devblogs.microsoft.com/devops/parallel-test-execution/ – Leo Liu Oct 02 '20 at 09:30
  • 1
    @LeoLiu-MSFT We have more than 6000 units tests, and even I agree that inside one particular assembly, we might have some cases that needs the same ressource system(and therefore are not able to be performed in a parallel manner), most of them are only testing one class of our code and you could even run 10x the same instance of test. I've read your article, but I can't find what should be helpfull about why it doesn't run in parallel? – J4N Oct 02 '20 at 15:02
  • @J4N, Yes, This is weird, I checked your settings and everything seems to be ok. Have you tested it locally without Azure devops? – Leo Liu Oct 12 '20 at 08:25

0 Answers0