How do Midje and Speclj compare?

Question

Both look reasonably good. I'd like to understand what each library is particularly good at or lacking, especially for testing of web applications.

Answer 1

I haven't used speclj, and I was the first author of Midje. One point that hasn't been mentioned by others is that Midje tries to exploit differences between functional and object-oriented languages.

One difference is immutability. Because most functions depend only on their inputs, not on contained state, the kind of truth statements you make about them are different in feel than their object-oriented counterparts. In OO testing, you make examples of the form: "given this history and these inputs, this method produces such and so."

It would seem that examples in a functional language would just be simpler ones: "given these inputs, this function returns such and so". But I don't think that's quite right. I think the other functions in the system play a role analogous to state/history: they're one of the things you're trying to get intellectual control over. Functions and their relationships are the things you want the tests to help you think clearly about.

For that reason, Midje is written under the assumption that a sweet development process involves saying:

• What do I want to be true of this function? In the world of the system, what's a good way to think of what this function does?
• In the process of doing that, what other functions would be useful---would capture an important part of the domain---and what truth statements do I want to make about them?

And then, in typical mockist style, you develop roughly top-down or outside-in, allowing for the inevitable iteration as you recover from mistakes or have better ideas.

The end result is to be a big pile of functions, with their interrelationships documented by the tests or (as Midje calls them) the "facts" about functions and the functions they depend on. Various people have commented that Midje has a Prolog/logic programming feel to it, and that's not an accident. As always, tests are examples, but Midje tries to make them read more like truth statements. That's the justification for its only actually innovative feature, metaconstants. Here's an example of them:

(fact "right changes the direction, but not the position"
  (right (snapshot north ...position...)) => (snapshot west ...position...)
  (right (snapshot east ...position...)) => (snapshot north ...position...)
  (right (snapshot south ...position...)) => (snapshot east ...position...)
  (right (snapshot west ...position...)) => (snapshot south ...position...))

In this case, the actual position is irrelevant to what's true about the function right, except that it never changes. The idea of a metaconstant is that it is a value about which nothing is known except what's explicitly stated in the test. Too often in tests, it's hard to tell what's essential and what's incidental. That has a number of bad effects: understanding, maintainability, etc. Metaconstants provide clarity. If it matters that a value is a map or record that contains the value 3 for key :a, you say that explicitly:

(fact
  (full-name ..person..) => "Brian Marick"
  (provided
     ..person.. =contains=> {:given-name "Brian", :family-name "Marick"}))

This test is explicit about what matters about people---and also explicit about what doesn't matter (anything but the two names).

In math terms, Midje is trying to let you make statements like "for all x where x..." while still being a test tool rather than a theorem prover.

This approach was inspired by "London-style" mock-heavy TDD of the sort described in Growing Object-Oriented Software, which is the approach I usually use in writing Ruby code. But it's turned out to have a pretty different feel, in a way that's hard to describe. But it's a feel that needs more tool support than just with-redefs.

The upshot is that Midje is in part an attempt to find a style of functional TDD that's not just a port of OO TDD. It tries to be a general-purpose tool, too, but it's semi-opinionated software. As Abraham Lincoln said, "Those who like this sort of thing will find this the sort of thing they like."

Answer 2

The biggest benefit of using Midje is that it provides focused abstractions for testing things without testing all of their parts, parts that often drag in the whole rest of the world.

If you have a function that involves calling a subsidiary function to generate a timestamp, putting something in a database or message queue, make an API request, caching something, logging something, etc, you want know that these world-involving function calls occurred (and sometimes how many times they occurred), however actually executing them is irrelevant to the function you are testing, and the called functions will often deserve having their own unit tests.

Say you have this in your code:

(defn timestamp [] (System/currentTimeMillis))

(defn important-message [x y] (log/warnf "Really important message about %s." x))

(defn contrived [x & y]
  (important-message x y)
  {:x x :timestamp (timestamp)})

Here is how you could test it with midje:

(ns foo.core-test
  (:require [midje.sweet :refer :all]
            [foo.core :as base]))

(fact
 (base/contrived 100) => {:x 100 :timestamp 1350526304739}
 (provided (base/timestamp) => 1350526304739
           (base/important-message 100 irrelevant) => anything :times 1))

This example is just a quick glimpse at what you can do with midje but demonstrates the essence of what it is good at. Here you can see there is very little extraneous complexity needed to express:

1. what the function should produce (despite the fact that what the timestamp function would produce would be different each time you call the function),
2. that the timestamp function and the logging function were called,
3. that the logging function was only called one time,
4. that the logging function received the expected first argument, and
5. that you don't care what the second argument it received was.

The main point I am trying to make with this example is that it's a very clean and compact way of expressing tests of complex code (and by complex I mean it has embedded parts that can be separated) in simple pieces rather than trying to test everything all at once. Testing everything all at once has its place, namely in integration testing.

I am admittedly biased because I actively use midje, whereas I have only looked at speclj, but my sense is that speclj is probably most attractive to people who have used the analogous Ruby library and find that way of thinking about tests ideal, based on that experience. That is a perfectly rspectable reason to chose a testing framework, and there are probably other nice things about it as well that hopefully others can comment on.

Answer 3

I'd definitely go with Speclj.

Speclj is simple to integrate and use. Its syntax is less flamboyant than Midje's. Speclj is based on RSpec to give you all the conforts that Ruby programmers are used to without losing the idiosyncrasies of Clojure.

And the auto runner in Speclj is great.

lein spec -a

Once you've used that for a while, you'll wonder how you ever got work done when you had to manually run tests.

Mocking is a non-issue since you can simply use with-redefs. @rplevy's example in Speclj would look like this.

(ns foo.core-spec
  (:require [speclj.core :refer :all ]
            [foo.core :as base]))

(describe "Core"
  (it "contrives 100"
    (let [message-params (atom nil)]
      (with-redefs [base/timestamp (fn [] 1350526304739)
                    base/important-message #(reset! message-params [%1 %2])]
        (should= {:x 100 :timestamp 1350526304739} (base/contrived 100))
        (should= 100 (first @message-params))))))

This bare-bones approach to mocking is to-the-point; no misdirection.

As for testing web apps, Speclj works fine. In fact Speclj support is build into Joodo.

disclaimer: I wrote Speclj

Answer 4

I'd say that Midje is especially good at creating a DSL for expressing stubbing and mocking. If you care about stubbing and mocking, and want to use it a lot, I'd choose Midje over Speclj, because it has abstractions for expressing those types of tests that are more concise than the approach slagyr offered in his answer.

Another option, if you want a more light-weight approach, is the Conjure stubbing/mocking library intended to be used with clojure.test.

Where Speclj shines is in being a lot like RSpec, having 'describe' and 'it' included... Midje can support nested facts actually, but not as elegantly as Speclj.

disclaimer: I'm a contributor to Midje and Conjure. :)

Answer 5

I would suggest Midje over Speclj
For speclj, I don't think if it has good support for mocks, the documentation also looks sparse as compared to Midje.

Also the syntax for Midje is better:

(foo :bar) => :result compared to (should= (foo :bar) :result)