3

we are starting with UnitTests now to raise our code quality a lot and (of course) for different other reasons. Sadly we are very late to the party, so we have classes with ~50 methods which are completly untested until now. And we have a lot of classes!

Is it possible / how is it possible to:
a) see which methods are untested?
b) force a developer who adds a method to a class to write a unit test for it (e.g. I achieve to test all 50 methods and tomorrow someone adds a 51. method)?
c) raise a warning or a error when a method is untested?

Rene Koch
  • 317
  • 3
  • 13
  • 1
    May be you should use reflection+Roslyn to get methods,who are untested,but i think you should write this code yourself – user2545071 Aug 24 '15 at 07:14
  • 1
    Are you using Continuous Integration? Do you have a Build Server? What metrics do you run on your build server script? Do you have gated check-ins? What policies govern gated check-ins? Once you can answer all those questions, you should have your answer. – Aron Aug 24 '15 at 07:14
  • 1
    You're looking for a code coverage tool – Sriram Sakthivel Aug 24 '15 at 07:17
  • @Aron: Yes, since 4 weeks we are using TeamCity as a CI and a Build Server. We don't have gated CheckIns yet, but it is planned soon. – Rene Koch Aug 24 '15 at 07:18
  • @ReneKoch why you delete the last question ? I was typing an answer – Aristos Aug 25 '15 at 10:51

3 Answers3

2

a) see which methods are untested?

This is fairly easy with a Code Coverage tool. There's one built into Visual Studio, but there are other options as well.

b) force a developer who adds a method to a class to write a unit test for it (e.g. I achieve to test all 50 methods and tomorrow someone adds a 51. method)?

Don't make hard rules based on coverage!

All experience (including mine) shows that if you set hard code coverage targets, the only result you'll get is that developers will begin to game the system. The result will be worse code.

Here's an example; consider this Reverse method:

public static string Reverse(string s)
{
    char[] charArray = s.ToCharArray();
    Array.Reverse(charArray);
    return new string(charArray);
}

You can write a single test to get 100 % coverage of this method. However, as it's given here, the implementation is brittle: what happens if s is null? The method is going to throw a NullReferenceException.

A better implementation would be to check s for null, and add a branch that handles that case, but unless you write a test for it, code coverage will drop.

If you have developers who don't want to write tests, but you demand a hard code coverage target, such developers will leave the Reverse function as is, instead of improving it.

I've seen such gaming of the system in the wild. It will happen if you institute a rule without buy-in from developers.

You'll need to teach developers how their jobs could become easier if they add good unit tests.

Once developers understand this, they'll add tests themselves, and you don't need any rules.

As long as developers don't understand this, the rule will only hurt you even more, because the 'tests' you'll get out of it will be written without understanding.

c) raise a warning or a error when a method is untested?

As explained above, code coverage is a poor tool if used for an absolute metric. On the other hand, it can be useful if you start monitoring trends.

In general, code coverage should increase until it reaches some plateau where it's impractical to increase it more.

Sometimes, it may even drop, which can be okay, as long as there's a good reason for it. Monitoring coverage trends will tell you when the coverage drops, and then you can investigate. Sometimes, there's a good reason (such as when someone adds a Humble Object), but sometimes, there isn't, and you'll need to talk to the developer(s) responsible for the drop.

Mark Seemann
  • 225,310
  • 48
  • 427
  • 736
1

To see which methods are untested there are tools for code coverage(either built-in or external like NCrunch, dotCover). It depends on your needs according results format, reports etc.

Those tools can be run either with VS or e.g. while building with build server(like TeamCity, Bamboo). Build can be set to fail when code coverage lowers(it is somehow answer to your c) question).

In my opinion you should force dev to write unit test by some plugins. It should be the part of your process, it can be pointed during code review. What is more - how you would like to know when to force to write a test? I mean - you don't know when the process of the development finishes so it is hard to tell someone "write your test right know".

EDIT: If you want some free code coverage tool for TC the one I know is OpenCover - you can easily find some information about plugin installation and usage.

kamil-mrzyglod
  • 4,948
  • 1
  • 20
  • 29
  • Thanks for the overview, we are using TeamCity since 4 weeks. Can you give me an overview about a free tool which integrates well with TC? – Rene Koch Aug 24 '15 at 07:19
1

That's funny, your section B "force a developer who adds.." was a question I posted just a few minutes ago, and I got so many down votes I removed the question, because apparently, it's not the right approach to force someone to write tests, it should be a "culture" you set in your organization.

So to your question (which is a bit wide, but still can get some direction):

a. You can use existing code coverage tools / libraries or implement your own to achieve this functionality, this is a good post which mentions a few. What can I use for good quality Code Coverage for C#/.NET?

You can build customization based on such libaries to assist with achieving answer to section c (raise a warning or a error when a method is untested) as well.

b.You should try implementing TDD if you can, and write the tests before you write the method, this way you won't have to force anything. If you can't do it since the method cannot be changed due to time and resource considerations, do TED (Test eventually developed).

True it won't be a unit test but still you'll have a test which has value.

Community
  • 1
  • 1
Y.S
  • 1,860
  • 3
  • 17
  • 30