26

My project has 1000+ unit tests that, in a local machine, all run in less than 10 seconds. But when they run on TFS Build, some tests run significantly slower than others. 3 of them run in about 1-2 minutes, other 4 in 5-30 seconds, and the others in fractions of a second. I've noticed that all those slower tests use fakes from Microsoft Fakes, and each one of them is the first to run in it's class. But a lot of the other tests also use fakes (some more intensively) and run in regular time. I would like to know what may be causing that slowdown and how can I fix it.

Edit: I've noticed that every slower test runs after a mockless test. Maybe that slowdown is caused by the initialization of the ShimsContext. In my test classes, the ShimsContext is created and disposed on TestInitialize and TestCleanup methods. Does that affects significantly the performance?

sshm
  • 269
  • 2
  • 4
  • @sshm did you find out why this was happening. My tests run in 20 seconds locally and currently i'm up to 35 minutes on the hosted build server! thanks Russ – RuSs Feb 16 '16 at 04:25
  • Is this a new slow down, or has it always been this way? A first guess would be that on the TFS Build it is logging the test results after each test completes and the round trip is killing you. I'd look at the trx file and see how long that thinks the tests took to run. Another common issue seems to be if you have intellitrace configured to run on all tests. There is then a lot of info that needs to be gathered. – Mike Aug 26 '18 at 13:23
  • Same issue here. We have 400 or so unit tests taking about 5 minutes to run locally and over an hour to run in TFS build. Very frustrating. – Craig Dec 09 '20 at 13:49

2 Answers2

1

First off all I would strongly recommend you move away from shims. They are a crutch, and aside from a very few scenarios, simply not needed. Design your code for testability, and you will find you can do without them.

Second, shims are not thread safe, and cannot be used concurrently safely. Likely this is why you are seeing the really slow run times.

Either your local is running things concurrently when it shouldn't (MS says it isn't safe, but not enforced), and the build server isn't.

Or the build server is trying to be parallel and it is causing issues.

Tweak your concurrency settings to disable it and see what effect that has on your runtime.

Tim
  • 2,878
  • 1
  • 14
  • 19
0

Refer following links : https://softwareengineering.stackexchange.com/questions/184834/how-do-we-make-unit-tests-run-fast

http://arlobelshee.com/the-no-mocks-book/

The links say that making tests fast can be hard. Decoupling is key. Mocks/fakes are OK, but one can do better by refactoring to make mocks/fakes unnecessary.

jo_Veera
  • 293
  • 3
  • 12
  • 2
    The microsoft developer community points out the slowness of unit tests. For them too the unit tests are running very slow on TFS Build even with better machine configuration. Refer following link : https://developercommunity.visualstudio.com/content/idea/351207/starting-then-running-all-unit-tests-in-a-solution.html – jo_Veera Mar 28 '19 at 06:03