28

I'm writing acceptance tests in Gherkin where I want to test for multiple changes in the UI of a web app based on an initial action. Here's an example:

        Scenario: Cancel editing a new text asset
            Given the user "test_user@fakedomain.com" is logged in
            When the user navigates to "/build/"
            And the user clicks the "Sandbox" link
            And the user inputs "Test story for canceling editing of a new text asset" for the "title" field
            And the user inputs "Test User" for the "byline" field 
            And the user inputs "My summary, so exciting!" for the "summary" textarea
            And the user clicks on "Untitled Section" in the section list
            And the user clicks the "Text" icon in the "center" container 
            And the user inputs the following text in the rich text editor:
                    """
                    Test text for asset. This is cool. 
                    """
            And the user clicks the "cancel" button
            Then the following text is not present: 
                    """
                    Test text for asset. This is cool. 
                    """
            And the "Image" icon is present
            And the "Text" icon is present
            When the user refreshes the browser 
            And the user clicks on "Untitled Section" in the section list
            Then the following text is not present:
                    """
                    Test text for asset. This is cool. 
                    """
            When the user opens the asset drawer
            Then the following text is not present:
                    """
                    Test text for asset. This is cool.
                    """

Note that there are multiple groups of When/Then steps, to test for responses of the initial action. While most implementations of steps ignore the prefix keyword, and I'm pretty sure that I can get this test to run, is there a better way to test the different outcomes? Is it better practice to write multiple scenarios with the same setup but different "Then" statements?

Geoffrey Hing
  • 1,575
  • 2
  • 15
  • 22
  • Hmm. Doesn't answer the question of whether its possible or not. Now onto more Google searching... – djangofan Oct 24 '16 at 20:18
  • Possible duplicate of [Is it acceptable to write a "Given When Then When Then" test in Gherkin?](https://stackoverflow.com/questions/12060011/is-it-acceptable-to-write-a-given-when-then-when-then-test-in-gherkin) – Dave Schweisguth Jul 21 '17 at 20:05
  • I disagree with the accepted answer; I added [my answer](https://stackoverflow.com/a/45245799/634576) to the question of which this is a duplicate. – Dave Schweisguth Jul 21 '17 at 20:09

2 Answers2

44

Remember that you should test only ONE behaviour/feature at a time. The rule of thumb is that you should use only one When step:

Given some state before action
  And some other state before action
  ...
When  only one action
Then  assert correct output
  And assert correct output
  ...

You see - only one line of When, without any Ands under When. If you use many When steps instead, you create test script, not a specification. Your tests will be hard to understand and you will notice that you add more and more steps when the underlying implementation changes.

You also need to keep underlying logic hidden, because you don't want to change it every time you change something irrelevant. Example:

And the user inputs "My summary, so exciting!" for the "summary" textarea

What if you change the summary field from a textarea to an input type? You have to change the scenario (maintenance nightmare) or left you scenario lying (worse than not having scenario). You should write instead:

When the user describes it as "so exciting!"

But still, the structure of the whole scenario is bad. Ask yourself the question: what I want to check? If I were a person that wants to understand the business logic of the feature I would like to see something like:

Scenario: Cancel editing a new text asset
  Given I edit the Sandbox details with some data
  When  I cancel editing
  Then  Sandox details should be empty

That's it!

How to achieve it? Move all irrelevant logic deeper, use the PageObject pattern etc. And read about Specification By Example

Werner Erasmus
  • 3,988
  • 17
  • 31
Michael Szymczak
  • 1,216
  • 13
  • 15
  • thanks for your helpful response. I hear what you're saying about making the specs reflect the core business logic of a feature and thereby making them more future-proof. Is it then inappropriate to use Features/Scenarios to test specific UI implementations around a feature? If so, what would be a better fit? I understand that in my example, there's some ambiguity between the boundary between the feature and a particular UI component. – Geoffrey Hing Oct 16 '13 at 18:04
  • 1. For UI testing use gherkin only if (because of the used tools, framework etc.) this is the fastest way to do it. 2. If you use gherkin to comunicate with others focus on user goals, not UI and use PageObjects to wrap the underlying logic. For the former use the the fastest tool - most of the time business people don't bother about differences between textareas and input fields. I for example use only PageObjects + Javascript unit tests to test UI logic, because in in our case this is the fastest approach. – Michael Szymczak Oct 17 '13 at 21:38
  • 1
    Gherkin is meant to cover your business rules, not your code. BDD is about application design, not application testing. If you have functional elements you need tested, then use a functional testing framework, or a comprehensive set of unit tests. – Greg Gauthier Aug 27 '15 at 18:27
  • @GregGauthier Forgive me my tardy reply to your comment. I agree with you. If you read my first point, I discourage from using Gherkin language as UI testing framework as well. I said that you should consider using it *only* when all other options are worse/slower, which nowadays is probably never the case. You can read my http://www.michaelszymczak.com/article/testing-business-rules.html series to see that I prefer using Gherkin only for specifying business rules. We are on the same page here. – Michael Szymczak Jan 15 '19 at 21:07
3

In contrast to the previous answer, there is no strict rule on how one can use define the steps. Cucumber official documentation at https://cucumber.io/docs/gherkin/reference/ recommends to use only one When due to the fact that only behavior is listed in one acceptance criteria. This is just a recommendation, but not a rule. And Cucumber documentation is mostly focused on the Acceptance criteria specification, but not the test automation. So, it is definitely possible to follow how it best suits your requirement when it comes to automation testing. I'd recommend using fewer "And" keywords by merging different steps. Below points I recommend (It is a recommendation, but not a rule :) ):

  1. Use only one Given, When and Then keywords in one flow of a scenario, use "And" keyword if you need to specify extra steps at the appropriate event
  2. If you come across using too many "And" keywords, try to merge multiple such steps

In reality, we cannot just use only one When in the test automation, because:

  1. Number of tests in the automation increases
  2. Total execution time for the automation goes up
  3. We may be doing a lot of redundant actions across multiple scenarios if we use only one When. Consider that you need to perform 5 steps as the initial condition for testing 10 different actions. Here, you will perform these 5 steps for 10 times when you use only one "When" - This consumes too much time, and causes the tests to be unstable, and causes more load on your application
  4. Due to the increased number of tests, we need to spend more time to analyze the results, more time to maintain

I also recommend testing the behavior based on the requirement. If your requirement is about verifying something in the "test area" rather than the "input area", your step should indicate that. Remember that if the requirement is changed, development is code changed, and hence the test automation code is also changed.

CMM
  • 543
  • 4
  • 16