97

I have just been asked by my company to rewrite a largish (50,000 single lines of code) Java application (a web app using JSP and servlets) in Clojure. Has anyone else got tips as to what I should watch out for?

Please bear in mind that I know both Java AND Clojure quite well.

Update

I did the rewrite and it went into production. It's quite strange as the rewrite ended up going so fast that it was done in about 6 weeks. Because a lot of functionality wasn't needed still it ended up more like 3000 lines of Clojure.

I hear they are happy with the system and its doing exactly what they wanted. The only downside is that the guy maintaining the system had to learn Clojure from scratch, and he was dragged into it kicking and screaming. I did get a call from him the other day saying he loved Lisp now though.. funny :)

Also, I should give a good mention to Vaadin. Using Vaadin probably accounted for as much of the time saved and shortness of the code as Clojure did.. Vaadin is still the top web framework I have ever used, although now I'm learning ClojureScript in anger! (Note that both Vaadin and ClojureScript use Google's GUI frameworks underneath the hood.)

Lii
  • 11,553
  • 8
  • 64
  • 88
yazz.com
  • 57,320
  • 66
  • 234
  • 385
  • 55
    I want to work for your company. – mtyaka Mar 08 '11 at 13:13
  • 4
    Well, some defence companies in Europe have started using Clojure (I heard). But can't remember any names :) – yazz.com Mar 08 '11 at 14:00
  • 1
    @Zubair: 50 000 Java LOC isn't anywhere near "largish". This is a very small project. I've got a 250KLOC to 300KLOC Java project here and it's medium-size... At best. – SyntaxT3rr0r Mar 09 '11 at 15:17
  • @Zubair: btw... How many lines do you think the Clojure code would end up to be? 75% size reduction? – SyntaxT3rr0r Mar 09 '11 at 15:21
  • 5
    Actually I maybe made a mistake by mentioning the size of the codebase as to be fair a reduction in code size is not the aim of the rewrite. The aim is twofold: 1) to have the codebase more understandable and thus cheaper to maintain code 2) Allow users of the tool to extend the product using a Clojure editor in the webbrowser (using eval and other dynamic Clojure goodness which is more costly to do in Java) – yazz.com Mar 10 '11 at 09:11
  • 300KLOC!!! Wow... How does that airplane ever fly! – yazz.com Mar 10 '11 at 09:13
  • 1
    Hey... just saw this older question and was wondering how the rewrite's going? – levand Jul 31 '11 at 22:34
  • I have updated the question with an update – yazz.com Sep 25 '11 at 09:25
  • Upvoted for that update. It's quite interesting and helpful to see the conclusion of cases like this. – efaj Aug 18 '13 at 06:24
  • 2
    The story is so juicy that it almost reads like a Clojure advertisement. Please please say this was real :) – Alex Dec 20 '15 at 18:35
  • This was a long time ago, like 4 years ago. The project added value to the company but I know that the company as a whole did not adopt Clojure, and went mostly for .Net. Unfortunately choosing a technology is not a business decision in a large enterprise. It is the technology that doesn't get a manager fired if it goes wrong that gets chosen – yazz.com Dec 23 '15 at 11:10

3 Answers3

81

The biggest "translational issue" will probably be going from a Java / OOP methodology to a Clojure / functional programming paradigm.

In particular, instead of having mutable state within objects, the "Clojure way" is to clearly separate out mutable state and develop pure (side-effect free) functions. You probably know all this already :-)

Anyway, this philosophy tends to lead towards something of a "bottom up" development style where you focus the initial efforts on building the right set of tools to solve your problem, then finally plug them together at the end. This might look something like this

  1. Identify key data structures and transform them to immutable Clojure map or record definitions. Don't be afraid to nest lots of immutable maps - they are very efficient thanks to Clojure's persistent data structures. Worth watching this video to learn more.

  2. Develop small libraries of pure, business logic oriented functions that operate on these immutable structures (e.g. "add an item to shopping cart"). You don't need to do all of these at once since it is easy to add more later, but it helps to do a few early on to facilitate testing and prove that your data structures are working..... either way at this point you can actually start writing useful stuff interactively at the REPL

  3. Separately develop data access routines that can persist these structures to/from the database or network or legacy Java code as needed. The reason to keep this very separate is that you don't want persistence logic tied up with your "business logic" functions. You might want to look at ClojureQL for this, though it's also pretty easy to wrap any Java persistence code that you like.

  4. Write unit tests (e.g. with clojure.test) that cover all the above. This is especially important in a dynamic language like Clojure since a) you don't have as much of a safety net from static type checking and b) it helps to be sure that your lower level constructs are working well before you build too much on top of them

  5. Decide how you want to use Clojure's reference types (vars, refs, agents and atoms) to manage each part mutable application-level state. They all work in a similar way but have different transactional/concurrency semantics depending on what you are trying to do. Refs are probably going to be your default choice - they allow you to implement "normal" STM transactional behaviour by wrapping any code in a (dosync ...) block.

  6. Select the right overall web framework - Clojure has quite a few already but I'd strongly recommend Ring - see this excellent video "One Ring To Bind Them" plus either Fleet or Enlive or Hiccup depending on your templating philosophy. Then use this to write your presentation layer (with functions like "translate this shopping cart into an appropriate HTML fragment")

  7. Finally, write your application using the above tools. If you've done the above steps properly, then this will actually be the easy bit because you will be able to build the entire application by appropriate composition of the various components with very little boilerplate.

This is roughly the sequence that I would attack the problem since it broadly represents the order of dependencies in your code, and hence is suitable for a "bottom up" development effort. Though of course in good agile / iterative style you'd probably find yourself pushing forward early to a demonstrable end product and then jumping back to earlier steps quite frequently to extend functionality or refactor as needed.

p.s. If you do follow the above approach, I'd be fascinated to hear how many lines of Clojure it takes to match the functionality of 50,000 lines of Java

Update: Since this post was originally written a couple of extra tools/libraries have emerged that are in the "must check out" category:

  • Noir - web framework that builds on top of Ring.
  • Korma - a very nice DSL for accessing SQL databases.
d0c
  • 515
  • 5
  • 7
mikera
  • 105,238
  • 25
  • 256
  • 415
  • 4
    Nitpick re: "Decide which of Clojure's STM reference types you want to use to manage each part mutable application-level state": There's only one STM reference type. The other IRefs don't involve STM. Otherwise looks like solid advice. –  Mar 08 '11 at 21:11
  • hmmm... I guess I count refs, agents and atoms as all being part of the Clojure STM/concurrency system. For example they all support validators, and agents are co-ordinated with commits of transactions. but I get your point, refs are the "primary" transactional model. will make a quick amendment. – mikera Mar 08 '11 at 21:23
  • 2
    If only I could give this more +1's. I watched both videos you referenced, and they were great. Thanks. – jdl Mar 09 '11 at 02:01
  • 1
    Now noir is deprecated. I suppose you have to mention about compojure instead of noir. – hsestupin Feb 22 '13 at 12:23
  • **One Ring To Bind Them link is broken!** – Adam Arold Apr 27 '14 at 20:50
5

What aspects of Java does your current project include? Logging, Database transactions, Declarative transactions/EJB, web layer (you mentioned JSP, servlets) etc. I have noticed the Clojure eco-system has various micro-frameworks and libraries with a goal to do one task, and do it well. I'd suggest evaluate libraries based on your need (and whether it would scale in large projects) and make an informed decision. (Disclaimer: I am the author of bitumenframework) Another thing to note is the build process - if you need a complex setup (dev, testing, staging, prod) you may have to split the project into modules and have the build process scripted for ease.

Felipe Augusto
  • 7,733
  • 10
  • 39
  • 73
Shantanu Kumar
  • 1,240
  • 1
  • 12
  • 14
  • Servlets, JSP, homebuilt persistence framework (10 years old) and pojos, but no EJB – yazz.com Mar 08 '11 at 20:46
  • 4
    Servlets can be easily replaced with Ring+Compojure IMHO, and JSPs can be replaced with perhaps StringTemplate (or FreeMarker/Velocity templates.) Persistence in Clojure is going to be different from Java. If you need relational mapping, you can look at Clj-Record and SQLRat (not very mature yet). ClojureQL supports only MySQL and PostgreSQL at the moment AFAICT. The current limitation in c.c.sql to disallow underscores in column names may come as a surprise. I think discussing the development aspects on the Clojure list as-and-when-required will be useful. Good luck on the project! – Shantanu Kumar Mar 08 '11 at 21:29
  • Since that project I made another application with Clojure and Clojurescript (nemcv.com) and I am using Ring now, tried ClojureQL, but switching to Korma for database access. You can see the latest work at https://github.com/zubairq/coils – yazz.com Aug 18 '13 at 09:22
4

I found the most difficult part was thinking about the database. Do some tests to find the right tools you want to use there.

LenW
  • 3,054
  • 1
  • 24
  • 25
  • 1
    Well I've tried clojureql for data access, but it is totally different from the Java style of database access which is all object based. Just out of interest what database access did you use for Java and what did you use with Clojure? – yazz.com Mar 08 '11 at 14:20
  • 1
    We ended up rethinking a lot of stuff and going with mongodb as the persistence of clojure data sturctures was very natural – LenW Mar 11 '11 at 12:42