7

We are currently running an OpenCover session, which is running the nunit3.console.exe.

Our command line is the following:

"C:\Program Files (x86)\OpenCover\OpenCover.Console.exe" -output:"%CD%\opencover.xml" -register:user -target:"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" -targetargs:"Solution\our-solution-file.sln --config=Debug --result=%CD%\TestResult.xml;format=nunit2"
exit 0

We were expecting this to be slower than our normal unit test due to the instrumentation in between, but not that much.

Without code coverage, the unit tests take something like 1h. And currently, with the code coverage, we already spent 3 days and 23hours, and we think we only executed less 10%.

Those results should be exported to SonarQube after.

Is there something we can do to improve the speed(except upgrading the computer running the test, which will probably be done anyway)?

Like having less detailed results, ... ? We are mostly interested by the code coverage, the duration and other stuff is not very interesting for us. Or even using another tool than OpenCover.

I don't know if this matters, but this line is executed by jenkins.

G. Ann - SonarSource Team
  • 22,346
  • 4
  • 40
  • 76
J4N
  • 19,480
  • 39
  • 187
  • 340
  • A slowdown of 60x is outright ridiculous. But this SO response suggests it is a property of OpenCover: http://stackoverflow.com/a/26225013/120163 The remark about using threads and queues is pretty surprising; those are very slow mechanisms to use if they are part of the runtime core of the tool. I'd expect a good test coverage tool to add 15-20% additional overhead to execution. . Semantic Designs (my company's) tools have this property. (See bio). – Ira Baxter Jan 31 '17 at 13:51

3 Answers3

1

By trying some things I did notice an huge improvement:

I excluded the tests assemblies of the openCover instrumentation, and now the performances are quite nice:

  • 1h06 only with UnitTests+SonarQube

  • 1h38 with OpenCover+UnitTests+SonarQube

This is quite acceptable for us.

By the way, how I did filter:

"C:\Program Files (x86)\OpenCover\OpenCover.Console.exe" -filter:"[*]* -[*.Test]*" -output:"%CD%\opencover.xml" -register:user -target:"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" -targetargs:"Solution\our-solution-file.sln --config=Debug --result=%CD%\TestResult.xml;format=nunit2"
exit 0
J4N
  • 19,480
  • 39
  • 187
  • 340
  • Well, I'm not sure what is wrong, but now the same command line takes again forever, maybe I got more ressources once due to the fact it is on a VM. – J4N Feb 07 '17 at 07:04
1

I know your post is now 5 years old but did you know there is a -threshold: option. Not entirely sure what it does but I suspect it breaks out of loops with large number of iterations.

I've added

-threshold:10

and now OpenCover speeds along, from hours and hours to a few minutes.

Nick
  • 358
  • 2
  • 13
0

Found this. Maybe it will help someone.

You could therefore try adding the -threshold option, e.g -threshold:100, this stops OpenCover from recording visits for a seq/branch point once it has been reached this in turn reduces the flood of information into the host from the profiler. This is useful if you care about a visit but not how many times - it doesn't play well with the -coverbytest feature because of the reduced recording but you may find it helps with the overall performance.

https://github.com/OpenCover/opencover/issues/268

Justin
  • 1