1

I am implementing a small API which send different requests to a server and works with the responses. I am using Moq and NUnit and try to implement everything in the Test First Manner. I am trying to encapsulate everything into Interfaces so I can mock them which works quite well. At some point my API is also tested by QA with real data (from the server) since the server is also being developed parallel it is sometimes not possible to tell where a bug occurs. Therefor I just wrote a test class with some unit tests which basically are just sending out the request to the server and checking is there is a response coming back. Something like that:

[Test]
    public void LoginShouldBeSuccesful()
    {
       result = webclient.UploadString(https://someURL.com, "POST", query);

       Assert.That(result,Is.Not.Null);
    }

This makes is easy for me to test in VS if the server is up and running, if some requests are already implemented and working and so on. (and it is fast than using some browser REST API plugin). However I added more and more tests which rely on different webrequests (which I already put into unit tests). So when I for example wanted to test a webrequest which deletes a user generated db entry I first had to run the Login unit test -> added the response (token) to the GenerateDBEntry unit test-> added the EntryID to the DeleteEntry request.

I know that this is against the principal of unit tests since they should run an be tested totally independent of each other, but I was wondering if there is a way to have them ordered and rely on each other (via return values) so that I can create test classes which are actual use cases. If possible I want to have one use case for each test class.

zlZimon
  • 2,334
  • 4
  • 21
  • 51

1 Answers1

1

I think it's useful to draw a distinction and use tools that are best suited for the job. What it looks like you are describing is specification testing, which is the realm of Behavior Driven Design (BDD).

A short list of C# BDD test suites include:

These are designed around testing your use cases, and helping you design use cases that are testable. The key difference being that BDD tools are geared for making sure the application as a whole works as designed. TDD tools are geared to make sure each part works as designed. The language, terminology, and concepts better support what you are trying to do.

NUnit limitations:

  • You cannot impose order. Any order you see in one tool is not guaranteed to work the same in another tool (and I've seen this bear out in practice). Some unit test runners sort in alphabetic order, and others keep the order defined in the class.
  • You cannot have tests depend on each other. For the same reason as bullet one.

NUnit and any of the XUnit style unit test frameworks (JUnit, etc) are very opinionated on this point. The design constraint that drives unit testing is that:

  • A unit test must be consistent, isolated and atomic (but not durable). I.e. any one unit test must be able to run on it's own, produce the same results every time, and it must not rely on or affect any other unit test.

Some unit test frameworks go to great pains to ensure these requirements, going so far as to dynamically load a DLL, run a test, unload it, and repeat with a new class loader just to ensure static values don't affect other tests.

I'm afraid that you'll have to do a lot of work in your set up and tear down to get the unit tests to behave the way you want. You may even have to artificially split up your tests among different classes to make sure the proper sequence of events happens in the order you expect.

It's just not the right tool for the job. That's why I recommended the BDD approach since you can impose order and dependencies using those tools.

Community
  • 1
  • 1
Berin Loritsch
  • 11,400
  • 4
  • 30
  • 57
  • yes I know those tools and the principle of BDD or ADD, I actually would like the exact other way around. Instead of having a scenario and let f.e. Specflow generate all unit tests I want to have my already existing unit tests gathered into a scenario (and for that they have to run in a specific order and are dependent of each other) – zlZimon Nov 12 '15 at 13:58
  • @zlZimon, added explanation as to why it's very impractical to do what you want with NUnit. – Berin Loritsch Nov 12 '15 at 14:07
  • thank you for detailed reply. I knew that my approach was not made for that but I was hoping that there was a "hacky" way to do that. Before the start of the project I was looking at Specflow but I actually thought that in the end only unit tests are coming out of it, I did not know that you also have a spefic order for the tests then. – zlZimon Nov 12 '15 at 15:30
  • Nunit now has an order attribute (https://github.com/nunit/docs/wiki/Order-Attribute). I still think you're right about using a BDD tool, but that bit about nunit's limitations isn't necessarily valid anymore. – senschen Nov 14 '16 at 15:40
  • @senschen, That's very interesting. The inclusion of that feature signifies a very distinct change in philosophy since the concept of **not** ordering tests was originally by design. The limitation wasn't that ordering tests _couldn't_ be done, but rather that it _shouldn't_ be done. – Berin Loritsch Nov 14 '16 at 19:20