18

I'm looking for a way to boost my team's productivity, and one way to do that would be to shorten the time it takes to compile & unit test & package & deploy our Java EE application which is getting bigger and bigger.

The trivial solution that I know of is to set up a powerful computer with N processors (N ~= num of developers) and a blazingly fast disk system and a lot of memory, and run everything on this computer and connect to it via X remotely. It would certainly be much faster than compiling on our laptops, but still cheaper and easier to maintain than to buy each developer his/her own supercomputer.

Is there another way to solve this problem? For example, could we run our IDEs locally and then tell it to remote compile java source? Can Netbeans / Eclipse / IntelliJ / etc. do this? Or is there a special tool that enables remote java compilation, also that makes use of multiple processors? It need not be free/open source.

Unfortunately our laptops MUST run a (company managed) Windows Vista, so another reason to go for the separate server computer is to let us use linux on it and finally get rid of the annoying managed environment.

EDIT: to sum up the answers so far, one way to shorten build times is to leave compilation for the developers individually (because compiling is supposed to be fast), skip running unit tests and hot-deploy (without packaging) to the container.

Then, when the developer decides to check his/her code in, a continuous integration server (such as Hudson) is triggered to clean & build & run tests & package & deploy.

SOLUTION: I've accepted Thorbjørn's answer since I think that's going to be the closest to which way I'm planning to proceed. Although out of curiosity I'm still interested in solving the original problem (=remote Java compiling)...

egbokul
  • 3,944
  • 7
  • 36
  • 54
  • @Gabor Kulcsar: I am not Java pro, but I must have heard of the term Just In Time compile for Java. So are you saying a slight change in some of your source code will make the entire app recompile? Or your app is too big for compile regularly? – Michael Mao Oct 01 '10 at 08:58
  • Do you really mean compile? I ask because compiling takes very little time. How many classes do you have? – Qwerky Oct 01 '10 at 09:06
  • Compile & unit test & package is the minimum requirement. Compilation takes the least time, however we have ~10 projects that depend on each other, and we use Maven to do the build. The outcome of the build is an EAR file, that takes ~2 minutes to build locally - plus the unit tests. Partial compilation often doesn't work, and we'd have to redo the packaging anyway... – egbokul Oct 01 '10 at 09:12
  • @Gabor - also wondering what JDK level you are compiling at. We ran into this http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6843751 because we were using SE 6 to compile (EE 5 on our server). We backed down to compiling at SE 5 to get around the bug until Oracle/Sun fixes it. In our case compilation went from ten minutes up to about two hours. The more classes and the bigger the classpath, the longer the compile. Just thought I'd share that here in case you are on SE 6. – Chris Aldrich Oct 01 '10 at 15:05
  • @Chris: we use JavaEE 6 full profile. But I'll check this bug, thanks for mentioning it! – egbokul Oct 02 '10 at 06:24
  • @Chris, considered building with the eclipse compiler? Very easy with ant. – Thorbjørn Ravn Andersen Oct 02 '10 at 13:24

6 Answers6

6

It's common to set up a build server , e.g. running hudson to do the compiling/packaging/unit-testing/deploying.

Though you'd likely still need the clients to at least perform a compile. Shifting to using a build server, you might need to change the work process too if you arn't using a build server now - e.g. if the goal is to take load off the client machines, your developers will check code in , automatic unit tests gets run, instead of running unit tests first, then checking in.

nos
  • 223,662
  • 58
  • 417
  • 506
  • 1
    We do have a build server, which does nightly builds. I've tried Hudson (and didn't like it), but I can't see how it would be of our help. How does it help me _before_ I check in the code I'm working on? – egbokul Oct 01 '10 at 09:15
  • It doesn't help you with the compilation, but it takes the load off your computers for running unit tests, packaging,deploying – nos Oct 01 '10 at 09:27
  • +1 for Hudson. @Gabor: Get the devs to do their work on branches mainly and only do nightly builds for the trunk. Moreover, you can configure Hudson to build when it notices a commit (with hysteresis for efficiency), and you can (independently) tell it whether to watch the trunk or a branch. The combination means that you get auto-builds in the places where the developers do most of their work, and defined nightly builds of the trunk (where you commit features only once they're believed to work, naturally). – Donal Fellows Oct 02 '10 at 06:38
6

You essentially need two workflows.

  1. The OFFICIAL build, which checks out the sources, builds the whole thing from scratch, runs all the unit tests, and then builds the bits which will eventually ship to the customer after testing.
  2. Developer hot-deploying after each source code change into the container the IDE knows about.

These two can actually be vastly different!

For the official build, get Jenkins up and running and tell it to watch your source repository and build whenever there is a change (and tell those who break the build). If you can get the big computer for building, use it for this purpose.

For the developers, look into a suitable container with very good IDE deployment options, and set that up for usage for each and every developer. This will VERY rapidly pay off! JBoss was previously very good for exactly this purpose.

And, no, I don't know of an efficient remote java compilation options, and I don't think this is what you should pursue for the developers.

See what Joel thinks about Build Servers: http://www.joelonsoftware.com/articles/fog0000000023.html

If you don't like Jenkins, plenty others exist.

(2016 edit: Hudson changed to Jenkins. See https://stackoverflow.com/a/4974032/53897 for the history behind the name change)

Community
  • 1
  • 1
Thorbjørn Ravn Andersen
  • 73,784
  • 33
  • 194
  • 347
  • I admit that deploying to Glassfish from Netbeans should be a breeze, but it doesn't work. Especially not if we use Maven and not the built-in Netbeans ANT. Maybe I shall rephrase my question and ask for an IDE with the best deployment option? – egbokul Oct 02 '10 at 06:16
  • Additional information: we decided to go for JavaEE 6 full (EJB3.1, MDBs, JPA2, JSF2 etc). Do you know of any IDE other than Netbeans that supports JavaEE 6 containers (eg. Glassfish 3) with hot-deployment? – egbokul Oct 02 '10 at 06:21
  • Java EE 6 is very nice. I use the official Glassfish plugin for Eclipse available from the Eclipse marketplace, which redeploys very fast. – Thorbjørn Ravn Andersen Oct 02 '10 at 10:21
3

You could mount each developer dir with ntfs on the powerful machine and then create External Tool Configuration in Eclipse (GUI access), that would be triggering build on external server.

Gadolin
  • 2,636
  • 3
  • 28
  • 33
3

JavaRebel can increase productivity also. It eliminates the need for redeployments.. You can recompile a single file and see the changes being applied directly on the server.

anonymous
  • 31
  • 1
2

When things start getting too big for efficient builds, it may be time to investigate breaking up your code into modules/JARs (how it breaks apart would depend on many project specifics and how your team tends to work). If you find a good setup, you can get away with less compiling (dont always need to rebuild the whole project) and more/quicker copying/jaring to get to the point where you can test new code.

chrismh
  • 131
  • 4
0

What your project need is a build system to do the building, testing and packaging for you. Hudson is a good example of such a continuous integration build system.

rsp
  • 23,135
  • 6
  • 55
  • 69