29

Seems like the internet doesn't have a definitive answer, or set of principles to help me answer the question. So I turn to the great folk on SO to help me find answers or guiding thoughts :)

SpecFlow is very useful for BDD in .NET. But when we talk about BDD are we just talking integration/acceptance tests, or are we also talking unit tests - a total replacement for TDD?

I've only used it on small projects, but I find that even for my unit tests, SpecFlow improves code documentation and thinking in terms of language. Converseley, I can't see the full code for a test in one place - as the steps are fragmented.

Now to you..........

EDIT: I forgot to mention that I see RSpec in the RoR community which uses BDD-style syntax for unit testing.

Nicko-Mctricko
  • 461
  • 5
  • 6
  • 2
    When counting the number of each type of test, you should think of a triangle. The base is unit, the middle is integration, and the top is UI testing. Int & UI tests are a yes but the sheer number of unit tests you are hopefully writing SHOULD make SpecFlow impractical for unit tests. – Brantley Blanchard Aug 01 '14 at 02:15

6 Answers6

32

I've recently started to use SpecFlow for my BDD testing, but also, I still use unit and integration tests.

Basically, I split the tests into seperate projects:

  • Specs
  • Integration
  • Unit

My unit tests are for testing a single method and do not perform any database calls, or external references whatsoever. I use integration tests for single method calls (maybe sometime two) which do interact with an external resources, such as a database, or web service, etc.

I use BDD to describe tests which mimick the business/domain requirements of the project. For example, I would have specs for the invoice generation feature of a project; or for working with a shopping basket. These tests follow the

As a user, I want, In order to

type of semantics.

My advise is to split your tests based on your needs. Avoid trying to perform unit testing using SpecFlow.

Jason Evans
  • 28,906
  • 14
  • 90
  • 154
  • That's exactly what we do too. You sure we don't work in the same company?? It works well. We also get our QA to write some of the BDD tests. – Ray Nov 11 '10 at 09:48
  • I agree, it works really well for me too. When I started out, I used to have all the tests in the same project - major mistake! – Jason Evans Nov 11 '10 at 09:53
  • Yeah, that's what I also do. Well to be honest, I do not have a distinction between specs and unit. But the philosophy is there, we all agree that we can use BDD - SpecFlow in this case - for our unit tests. Unit tests the client doesn't even care about, yet we still use the Cucumber syntax for developer only tests. – Nicko-Mctricko Nov 11 '10 at 11:25
  • Actually I mis-read your comment. Looks like you don't think using BDD for unit tests is a good idea. – Nicko-Mctricko Nov 11 '10 at 14:48
  • I known it is an old post, but I'm wondering if you, after working a long time with it, still think that BDD is not suitable for unit-tests? I agree that acceptance, integration and unit tests should be separated. But why shouldn't we describe the behavior of our units with BDD? I use BDD (by naming test classes and method with the given, then, should approach) for all my tests, but I'm currently investigating into specflow to define my tests. Any ideas welcome... – Tim Cools Mar 26 '12 at 13:39
  • 3
    You know, I've been thinking about this off and on, and overall I still wouldn't use BDD for unit tests. I've found myself using the context/specification approach eg MSpec, or NSpec for unit tests. This approach still allows descriptive test naming and gives clear documentation on how the "unit" is expected to work. For actual features of a projects, I'd still use BDD. – Jason Evans Mar 27 '12 at 08:21
  • @Jason Evans,+1. Can Specflow be used to test a domain model in isolation or should it be used to test all layers i.e. UI, infrastructure etc together? – w0051977 Mar 09 '18 at 08:26
  • I would test a domain model using tests written in NUnit/xUnit etc, not Specflow. My preference would be to use Specflow to test the acceptance criteria for a features/user stories. It feels more natural to use Specflow that way. – Jason Evans Mar 09 '18 at 08:58
  • @Jason Evans, I upvoted this on 09/03. What do you call your unit testing namespace? Is it: MyCompany.MyProduct.Feature.Unit? – w0051977 Mar 26 '18 at 07:41
  • I tend to be a bit different to your example. I use `MyCompany.MyProduct.Tests.Unit` or `MyCompany.MyProduct.Tests.Acceptance` for BDD tests. – Jason Evans Mar 26 '18 at 14:09
6

We have started using Specflow even for our unit tests.

The main reason (and benefit) for this is that we find that it forces you to write the tests from a behavior point of view, which in turn forces you to write in a more implementation agnostic way and this ultimately results in tests which are less brittle and more refactoring friendly.

Sure this can also be done with standard unit testing frameworks, but you aren't guided that way as easily as we have found we are using specflow and the gherkin syntax.

There is some overhead setting things up for specflow, but we find this is quickly repaid when you have quite a few tests (due to the significant step reusability that you can get with specflow) or you need refactor your implementation.

Plus you get nice readable specs that are easy for newcomers to the team to understand.

Sam Holder
  • 32,535
  • 13
  • 101
  • 181
3

Given:

  • Unit tests are test of (small) "units of code"
  • The customer of most “units of codes” are other programmers.
  • Part of the reason for having a unit test is to provide an example of how to call the code.

Therefore:

  • Clearly unit tests should normally be written in the programming language that the users of the “unit of code” will be calling it with.

However:

  • Sometimes data tables are needed to setup the conditions a unit test runs in.
  • Most unit test frameworks are not good at using tables of data.

Therefore:

  • Specflow may be the best option for some unit test, but should not be your default choose.
Ian Ringrose
  • 51,220
  • 55
  • 213
  • 317
  • Is this still relevant reason because not unit-test frameworks like MSTest, MSTest2 support data driven tests very well. – Nithin B Aug 03 '22 at 20:41
  • @NithinB That depends who you wish to be able to read the tests, and how many data tables need to be setup for the test to work. Thinking of a mocked data access layer that needs the contents of a few database tables defining. – Ian Ringrose Aug 04 '22 at 19:53
2

I see it as an integration testing which mean it doesn't replace your unit test cases written as part of your TDD process. Someone will have different opinion about this. IMHO unit test case only test the methods/functions and all the dependencies should be mocked and injected. When in it comes to integration testing, you will be injecting real dependencies instead of mocked one. You could do the same integration testing with any of the unit testing frameworks, but the BDD provides you cleaner way of explaining the integration test use case in a Domain Specific Language which is a plain English(or any localized language).

Ta,
Rajeesh

Rajeesh
  • 4,377
  • 4
  • 26
  • 29
1

I used specflow for BDD testing on two different good sized applications. Once we worked through the kinks of the sentence naming conventions, it worked out pretty good. BA's and QA's, and even interns could write BDD tests for the application.

However, I ALSO used it for unit tests. Heresy! I can hear some of you scream. However, there were VERY good reasons for it. The system was responsible for making many calculations or determinations based off a lot of different data. With lots of unit tests that require all this data to be input for test purposes, it makes it a LOT easier to manage that data used for the unit tests via the table format provided by specflow. Effectively mocking the data repository in table format, allowing the different components to be vigorously tested.

I don't know if I would do it in every case, but in the ones I used it for, it made laying out the volumes of data necessary for for performing the unit tests so much easier and clearer.

Keith
  • 1,119
  • 2
  • 12
  • 23
0

In the end we are trying to deliver to the customer exactly what the customer wants and as such I really don't see the need to write unit tests in addition to SpecFlow. After all, it exercises the same code base. I am fairly new to BDD/ATDD/TDD but other than being "complete" and strictly adhering to TDD I'm finding it unnecessary to write more unit tests.

Now I suppose if the team was dispersed and the developer was not able to run the entire application then separate unit tests would be necessary but where the developer(s) has access to the entire code base and is able to run the application, then why bother write more tests.

Nikhil
  • 16,194
  • 20
  • 64
  • 81
kmullings
  • 117
  • 2
  • 6