How do I get better feedback from Clojure errors?

Question

I find it very difficult to debug the Clojure errors I have in my code compared to all the other programming languages I've used. My primary programming language is Java and I'm very new to Clojure. The majority of my time writing Clojure is spent trying to figure out, "Why am I getting this error?" and I'd like to change that. I'm using CounterClockWise as my primary IDE. I don't know how to use Emacs (yet?).

Here's an example:

(ns cljsandbox.core)

(def l [1 2 3 1])

(defn foo
  [l]
  (->> l
    (group-by identity)
    ;vals  ;commented out to show my intent
    (map #(reduce + %))))

Here, I mistakenly thought that group-by returns a list of lists, but it actually returns a map of <key, list<value>> or however you'd phrase it in Java terms. This gives an error message that says:

ClassCastException clojure.lang.PersistentVector cannot be cast to java.lang.Number clojure.lang.Numbers.add (Numbers.java:126)

This isn't very helpful because there isn't a stack trace. If I type (e) it says:

java.lang.ClassCastException: clojure.lang.PersistentVector cannot be cast to java.lang.Number
 at clojure.lang.Numbers.add (Numbers.java:126)
    clojure.core$_PLUS_.invoke (core.clj:944)
    clojure.core.protocols/fn (protocols.clj:69)
    clojure.core.protocols$fn__5979$G__5974__5992.invoke (protocols.clj:13)
    clojure.core$reduce.invoke (core.clj:6175)
    cljsandbox.core$foo$fn__1599.invoke (core.clj:10)
    clojure.core$map$fn__4207.invoke (core.clj:2487)
    clojure.lang.LazySeq.sval (LazySeq.java:42)

I have no idea how I can go from this error message to understanding, "You thought you were passing in a list of lists into map but you were really passing in a map datatype". The stack trace shows the problem was reported inside of reduce, not inside of group-by, but IMO, that is not where I as a human made my mistake. That's just where the program discovered a mistake had been made.

Issues like these can take me 15+ minutes to resolve. How can I make this take less time?

---

I know it's too much to expect a dynamic language to catch these errors. But, I feel like the error messages of other dynamic languages like javascript are much more helpful.

I'm getting pretty desperate here, because I've been coding in clojure for 1-2 months now and I feel like I should have a better handle on figuring out these problems. I tried using :pre/:post on functions but that has some problems

1. The reporting on :pre/:post kind of sucks. It only prints out literally what you test. So unless you put a lot of effort into it, the error message isn't helpful.
2. This doesn't feel very idiomatic. The only code I've seen that uses :pre/:post are articles that explain how to use :pre/:post.
3. It's a real pain to pull out the steps of a threading macro into their own defns so that I can put the :pre/:post in them.
4. If I followed this practice religiously, I think my code could become as verbose as Java. I'd be reinventing the type system by hand.

I've gotten to the point where I pepper my code with safety checks like this:

(when (= next-url url)
            (throw (IllegalStateException. (str "The next url and the current url are the same " url))))      
(when-not (every? map? posts-list)
            (throw (IllegalStateException. "parsed-html->posts must return a list of {:post post :source-url source-url}")))

Which only fixes that first bullet point.

I feel like either

1. I've got a development process that's very, very wrong and I don't know it
2. There's some debugging tool/library out there that I don't know about that everyone else does
3. Everyone else is having problems like this and it's Clojure's dirty little secret / Everyone else is used to dynamic languages and expects to go through the same effort I am going through to resolve errors
4. CounterClockWise has some bug that's making my life way harder than it needs to be
5. I'm supposed to be writing a lot more unit tests for my Clojure code than I do for my Java code. Even if I'm writing throwaway code.

Answer 1

In this particular instance, discovering the source of the problem is easy:

1. We've got a function to be applied to a known vector of items. We're also expecting a particular result.

2. Applying the function results in a problem. Let's peek inside the function then; it happens to be a ->> pipeline.

3. The most straightforward way to diagnose the problem is to leave off some of the final stages of the pipeline to see if the intermediate stages in the transformation are as we expect.

Doing 3. is particularly straightforward at the REPL; one approach is to def the input and the intermediate results to temporary Vars, another is to use *1, *2 and *3. (If the pipeline is long or the computations take a lot of time, I'd recommend doing a temporary def at least once every few steps, otherwise the *ns might suffice.)

In other cases, you'd do something slightly different, but in any case breaking down the work into manageable chunks to be played with at the REPL is key. Of course familiarity with Clojure's sequence and collection libraries speeds the process up quite a bit; but then playing with them in the context of small chunks of an actual task you're working on is one of the better ways to learn about them.

Answer 2

The best way to make sense of clojure exceptions as of now (untill probably we have clojure in clojure) is to understand that clojure is implemented using Java classes,interfaces etc. So whenever you get any such exception try to map the classes/interfaces mentioned in the exception to clojure concepts.

For ex: In your current exception it can be easily inferred that clojure.lang.PersistentVector was being tried to type cast to java.lang.Number in the method clojure.lang.Numbers.add. From this information you can look into your code and intuitively figure out where you are using add i.e + in your code and then diagnose that problem by the fact that somehow this + is getting vector as parameter instead of number.

Answer 3

I find the clojure.tools.logging/spy macro very useful for debugging. It prints out the wrapped expression as well as its value. If setting up clojure.tools.logging isn't something you want to do right now (it requires the normal Java logging configurations) you can use this:

(defmacro spy
  [& body]
  `(let [x# ~@body]
     (printf "=> %s = %s\n" (first '~body) x#)
     x#))

*keep in mind that code above does not print out values of a lazy seq if it hasn't been realized. You can vec a lazy seq to force its realization - not recommended for infinite seqs.

Unfortunately, I haven't found a good way to use the spy macro within a threading macro, but it should suffice for most other cases.

Answer 4

Dynalint might be worth looking into. It wraps function calls with extra checks that hurt performance, but provide better error messages.

It's doesn't seem to be a very mature project, and hasn't been updated for a year, but it already makes some progress to improve error messages. Also, it is on the list for possible GSoC 2015 projects, so we might see a big improvement soon!