1

I'm currently learning/testing BDD using SpecFlow, and it works great!

Before I choose to ask my question, I have read this one, and I felt like I had to ask my question despite the fact that the same problem is addressed, because of the Exception scenario which is not mentioned.

I'm actually testing this scenario:

Scenario: I should get an error whenever I try to remove an item from an empty stack
    Given I have an empty stack
    When  I pop from it
    Then  I should get an error

public class StackBehaviour {
    public void GivenIHaveAnEmptyStack() { stack = new CustomStack<string>(); }

    // This will throw whenever called!
    // So the Then method will never be run!
    // I feel like I should just put a comment which says why it's empty,
    // allowing a fellow programmer to understand the exact intention.
    public void WhenIPopFromIt() { stack.Pop(); }

    // It is here that it verifies whether the CustomStack meets the expected behaviour.
    public void ThenIShouldGetAnError() {
        Assert.Throws<IndexOutOfRangeException>(delegate {
            stack.Pop(); 
        });
    }

    private CustomStack<string> stack;
}

public class CustomStack<T> {
    public T Pop() { 
        if (stack.Count == 0) 
            throw new IndexOutOfRangeException("Cannot pop from an empty stack!");
        T item = stack[stack.Count-1];
        stack.RemoveAt(stack.Count-1);
        return item;
    }

    private ArrayList stack = new ArrayList();
}

I think that leaving a comment in the When method is correct, so that the business requirement doesn't lack any information, and on the code behind, I put it clear what my intention is exactly by commenting.

What do you think? Any other ideas why I shouldn't make it?

Community
  • 1
  • 1
Will Marcouiller
  • 23,773
  • 22
  • 96
  • 162
  • Both provided answers are GREAT! I now have a hard time choosing which to accept as the very answer of my question, since these address the problem differently, and both answers could be as good as each other. First, I know I want an error, so just code the `When` accordingly. I totally agree! Second, technical matters such as pushing a null on a stack shouldn't matter the stakeholder, as it is more likely a technical concern, and this draws the line between BDD and TDD. I totally agree! – Will Marcouiller Dec 13 '13 at 22:01
  • @Alski answer does show how SpecFlow can be used to test this CustomStack, however I still feel that that using a BDD framework to test a single component in isolation is excessive. BDD is generally used to describe how a user interacts with a system; in your example a user would not directly pop an item off the stack, instead they'd execute an action in the application which makes use of the CustomStack which would then trigger the popping of the stack. Hence the reason why I think CustomStack would be better tested using TDD and unit tests. It'll be interesting to see what Alski thinks. – Ben Smith Dec 13 '13 at 23:11
  • I totally agree, Fresh. In the real world, BDD stands for testing the behaviour of the system as a whole. Because of it, strictly BDD speaking, your answer is the best for a real solution. Aside, Alski's answer fulfills the tutorial's need, that is, being able to use both the `When`, where the action occurs and the `Then`, where it shall be tested against the expected value/behaviour. My dilemma is all around what Alski's answer suggests, which solve my problem, and indeed we see where the line between BDD and TDD stands, because in fact, these exceptions don't belong to BDD but to TDD. – Will Marcouiller Dec 14 '13 at 04:18
  • I've had a journey in learning to use BDD and how not to use it and recently have moved to a team that shares my views. However I would definitely suggest that you need to go through a similar journey yourself to find the correct balance between BDD and TDD. Just telling you wont help you understand why necessarily. – AlSki Dec 14 '13 at 10:36
  • I started by using BDD a lot. Initially I almost used it exclusively, replacing unit testing just like you are. Later I started to see that NUnit tests could be quicker to write so started trying to build testing frameworks that combined GWT steps to make unit tests, but that was a fruitless exercise that simply generated more testing code. – AlSki Dec 14 '13 at 10:39
  • Since then I've consulted at a large firm on the basis of my experience and advised them improving there product testing. I followed all the best practices and worked with the BAs to develop from feature specifications into a large framework, only to see it fall by the wayside as business priorities trumped quality. – AlSki Dec 14 '13 at 10:43
  • I now work in a team that has the management support and BAs deliver feature specifications in gherkin format. Some features we rewrite as unit tests in //given //when //then sections. Otherwise we implement as proper Specflow tests. – AlSki Dec 14 '13 at 10:46
  • This approach is simply the product of trying to find balance between getting the job done on time and providing quality tests. Our SpecFlow tests are a major headache as they tend to be very sensitive to change and constantly need tweaking. Almost any new feature tends to break about a 100 tests, but the unit tests can often be fixed almost instantly, where as the gherkins require about 30 mins just to run, as by nature they fire up a client and interact with the UI in order to send queries to a server. – AlSki Dec 14 '13 at 10:52
  • But I still recommend doing some Integration testing and believe SpecFlow is the best tool in .Net for capturing proper conversations with the business. – AlSki Dec 14 '13 at 10:53
  • @alski great feedback. It's good hearing about your real world experiences with BDD and TDD. My experiences are very similar. I try to always use BDD and then revert to using TDD as part of the BDD process, but it definetly takes time to learn! – Ben Smith Dec 14 '13 at 21:53
  • Thank you both @Fresh and @Alski! I get the way to use both the `When` and the `Then` in the case of an exception is expected. It might be expected as well from a system, so I accepted Alski's answer becuase he shows the way I might catch the expected exception and then assert it in the `Then` method. Aside, I understand that for the sake of a stack, this should be unit tested instead. You both provided valuable information. Thank you very much to both of you! – Will Marcouiller Dec 17 '13 at 14:45

2 Answers2

2

There is another trick that you can use where the bindings help make the feature's meaning a little clearer. In this case, let's start at the end.

Then I should get an error

Your problem is a basically here, you know want an error, but you don't know how to get it. In fact you've already missed it, the error has already occurred in the When step, and in your code example, you moved the action into the then step just so you could get the error.

But what if keep the when performing the action amd re-express our then to reflect what really happens

Then I should have had an error

Seems a trivial change but now our feature reflects that the error should have been associated with the When step and we are simply evaluating it later, and that is something we can code. We just need something to remember the error in the when and deliver it to the then.

private Exception errorFromWhen = null;

public void WhenIPopFromIt() 
{ 
  try
  {
    stack.Pop(); 
  }
  catch(Exception ex)
  {
     errorFromWhen = ex;
  }
}

public void ThenIShouldGetAnError() 
{
   errorFromWhen.ShouldNotBeNull();
   errorFromWhen.ShouldBe<IndexOutOfRangeException>();
}

SpecFlow has absolutely no problems with this, in fact due to its mini Dependency injection system, you can even pass this state between binding classes if necessary.

AlSki
  • 6,868
  • 1
  • 26
  • 39
1

May a scenario not have a When in BDD?

In Specflow, neither given, when or then are mandatory.

However in your example, I don't believe this is a good use of Specflow and BDD. In this answer here Marcus states:

"BDD is about ensuring that we're building the right thing, TDD is about ensuring that we're building it right."

In your example the scope of what is being tested i.e. the CustomStack, should be tested via TDD. It is the end solution that makes use of the CustomStack should be tested via BDD (and hence SpecFlow) e.g. if this CustomStack was being exercised via a certain action on a website.

This answer is also pertinent to your question.

Community
  • 1
  • 1
Ben Smith
  • 19,589
  • 6
  • 65
  • 93
  • +1 Thanks for these insights. I shall take for granted that a unit tests project is still pertinent though the using also BDD. This draws the line not to cross between BDD and TDD. Aside, I was following this interesting article which tests the behaviour of a stack, let's say the stakeholder wants a stack object, so I wonder what line not to cross. http://www.ibm.com/developerworks/java/library/j-cq09187/index.html – Will Marcouiller Dec 13 '13 at 21:57
  • Thanks for your answer. I really learned a lot with it. Please see my last comment associated with my question so that you may know how come I did accept Alski's answer. If you feel like there's something wrong with my judgement, please feel free to say so and I shall consider to revisit my acceptance. – Will Marcouiller Dec 17 '13 at 14:47