1

Edit: This question seems to be related

This question is not about the differences between functional and acceptance tests! Almost all the info I could find on the web just explains the difference between them. I know that functional tests (FT's) address fringe conditions and bug scenarios, whereas Acceptance Tests (AT's) address business requirements. I achieve both with SpecFlow.

I am having some trouble wrapping my head around the separation of the two, from a project structure/hierarchy perspective. Currently, I have one unit-test project with an AcceptanceTests folder, and a FunctionalTests folder. All my step definitions are jumbled together in a StepDefinitions folder.

I find I'm having to repeat myself a lot and that the MsTest pane just mixes everything together when I group on Traits. I want to establish what the industry standard is out there, so I have five questions:

  • Do I repeat the "In order to... as a... I want to..." story in a separate feature-file for AT's and FT's?

  • Do I repeat all the AT's scenarios in the FT's as well, or only fringe condition scenarios?

  • Should I keep the AT's and the FT's in their own namespaces and/or their own projects?

  • Should I try to call the FT's scenario step methods from the AT's step definitions, seeing as the grunt-work is being done by the FT's anyway?

  • Any advice about my current setup (e.g. is it overkill to do both?) is welcome.

Community
  • 1
  • 1
Riegardt Steyn
  • 5,431
  • 2
  • 34
  • 49

1 Answers1

1

It's overkill to do both as SpecFlow features.

Acceptance tests are very valuable, but they take a lot of work to write and maintain and a long time to run, so you want to have as few of them as possible while still capturing all of your important use cases.

Instead, test your "fringe conditions and bug scenarios" with a unit testing tool such as NUnit.

You may find that for a given feature you want a single scenario in which the user makes a mistake so that you specify how your error handling works. But you definitely don't want a scenario for every specific error; instead, capture those in unit tests of your validation code.

Dave Schweisguth
  • 36,475
  • 10
  • 98
  • 121
  • Thank you Dave. Your concern over the time to run seems to suggest that acceptance tests should be integration tests. Do you run your acceptance tests against live databases, file systems & web services? I thought one should mock things out for acceptance tests; have I been understanding this all wrong? – Riegardt Steyn Feb 07 '16 at 11:14
  • 1
    Yes, you want your acceptance tests to run against as much of the system as possible. No, you don't want to mock in them, except where you have to to protect yourself from impractically slow or unreliable third-party services. But real (not production) databases and filesystems for sure. – Dave Schweisguth Feb 07 '16 at 13:13