14

I've used lettuce for python in the past. It is a simple BDD framework where specs are written in an external plain text file. Implementation uses regex to identify each step, proving reusable code for each sentence in the specification.

Using scala, either with specs2 or scalatest I'm being forced to write the the specification alongside the implementation, making it impossible to reuse the implementation in another test (sure, we could implement it in a function somewhere) and making it impossible to separate the test implementation from the specification itself (something that I used to do, providing acceptance tests to clients for validation).

Concluding, I raise my question: Considering the importance of validating tests by clients, is there a way in BDD frameworks for scala to load the tests from an external file, raising an exception if a sentence in the test is not implemented yet and executing the test normally if all sentences have been implemented?

tiagoboldt
  • 2,426
  • 1
  • 24
  • 30
  • I've already searched for something like that in Scala's world. Unfortunately, it seems there's no equivalent of Cucumber in Ruby world for instance, where we can have a totally independent file containing the specifications (Given, When, Then ...). You could use regexp parsing from Specs2 (as the documentation evokes) and load a pure txt file (scenarios) at the beginning of your spec test. But you would never have a "control" of which sentence hasn't been tested, according to me. – Mik378 Mar 13 '13 at 15:37
  • 8
    You just said 'text file' 'regexp' & 'reusable' in one sentence and then named the scala version ugly? #ImLost – Jens Schauder Mar 13 '13 at 16:04

4 Answers4

12

I've just discovered a cucumber plugin for sbt. Tests would be implemented under test/scala and specifications would be kept in test/resources as plain txt files. I'm just not sure on how reliable the library is and if it will have support in the future.

Edit: The above is a wrapper for the following plugin wich solves perfectly the problem and supports Scala. https://github.com/cucumber/cucumber-jvm

tiagoboldt
  • 2,426
  • 1
  • 24
  • 30
  • 4
    Many months later, I've ended up using scalatest exclusively – tiagoboldt Apr 01 '14 at 15:43
  • @tiagoboldt - why? Did you find a way to overcome the problems you raised in your original question by only using scalatest? – gordonm Feb 23 '16 at 12:02
  • @gordonm it was many years ago, I can't remember the exact reason, but I believe I've decided to go with scalatest despite the more complex syntax instead of pure gherkin to depend on a single lib for this. – tiagoboldt Feb 23 '16 at 12:46
9

This is all about trade-offs. The cucumber-style of specifications is great because it is pure text, that easily editable and readable by non-coders.

However they are also pretty rigid as specifications because they impose a strict format based on features and Given-When-Then. In specs2 for example we can write any text we want and annotate only the lines which are meant to be actions on the system or verification. The drawback is that the text becomes annotated and that pending must be explicitly specified to indicate what hasn't been implemented yet. Also the annotation is just a reference to some code, living somewhere, and you can of course use the usual programming techniques to get reusability.

BTW, the link above is an interesting example of trade-off: in this file, the first spec is "uglier" but there are more compile-time checks that the When step uses the information from a Given step or that we don't have a sequence of Then -> When steps. The second specification is nicer but also more error-prone.

Then there is the issue of maintaining the regular expressions. If there is a strict separation between the people writing the features and the people implementing them, then it's very easy to break the implementation even if nothing substantial changes.

Finally, there is the question of version control. Who owns the document? How can we be sure that the code is in sync with the spec? Who refactors the specification when required?

There is no, by far, perfect solution. My own conclusion is that BDD artifacts should be in the hand of developers and verified by the other stakeholders, reading the code directly if it's readable or reading an html/pdf output. And if the BDD artifacts are owned by developers they might as well use their own tools to make their life easier with verification (using a compiler when possible) and maintenance (using automated refactorings).

Eric
  • 15,494
  • 38
  • 61
  • The specs2 GivenWhenThen example linked above was recently removed from specs2, I think in preference to [this](https://github.com/etorreborre/specs2/blob/master/src/test/scala/examples/GWTSpec.scala) example which shows the new GWT trait described in [here](http://etorreborre.github.io/specs2/guide/org.specs2.guide.structure.GivenWhenThenPage.html). – Eric Drechsel Jun 04 '13 at 17:50
6

You said yourself that it is easy to make the implementation reusable by the normal methods Scala provides for this kind of stuf (methods, functions, traits, classes, types ...), so there isn't really a problem there.

If you want to give a version without code to your customer, you can still give them the code files, and if they can't ignore a little syntax, you probably could write a custom reporter writing all the text out to a file, maybe even formatted with as html or something.

Another option would be to use JBehave or any other JVM based framework, they should work with Scala without a problem.

Jens Schauder
  • 77,657
  • 34
  • 181
  • 348
  • 1
    "and if they can't ignore a little syntax" => Unfortunately, customers are, most of the time, afraid of any words related to coding. I've even met some customers who found the three poor words: "Given, When, Then" a huge burden... I find that more than stupid but imagine the reaction if I include scala code, even thin, in specs.. :) – Mik378 Mar 13 '13 at 16:16
  • 1
    Elaborating on what Mik378 said above, having the features in separate files could provide us with the ability to evolve them in isolation from the code. Tests would fail if a new feature was introduced externally, with well written acceptance tests being enough documentation for the dev team to implement the new feature. Let me remind you that those who write acceptance tests do not need to know about programming. – tiagoboldt Mar 13 '13 at 17:23
  • @tiagoboldt Totally agree. – Mik378 Mar 13 '13 at 19:22
0

Eric's main design criteria was sustainability of executable specification development (through refactoring) and not initial convenience due to "beauty" of simple text.

see http://etorreborre.github.io/specs2/

The features of specs2 are:

  1. Concurrent execution of examples by default
  2. ScalaCheck properties
  3. Mocks with Mockito
  4. Data tables
  5. AutoExamples, where the source code is extracted to describe the example
  6. A rich library of matchers
  7. Easy to create and compose
  8. Usable with must and should
  9. Returning "functional" results or throwing exceptions
  10. Reusable outside of specs2 (in JUnit tests for example)
  11. Forms for writing Fitnesse-like specifications (with Markdown markup)
  12. Html reporting to create documentation for acceptance tests, to create a User Guide
  13. Snippets for documenting APIs with always up-to-date code
  14. Integration with sbt and JUnit tools (maven, IDEs,...)

Specs2 is quite impressive in both design and implementation. If you look closely you will see the DSL can be extended while you keep the typesafe-ty and strong command of domain code under development.

He who leaves aside the "is more ugly" argument and tries this seriously will find power. Checkout the structured forms and snippets

SemanticBeeng
  • 937
  • 1
  • 9
  • 15