compiling 10000 of java project simultaneously

Question

Any one know how to compiling and run huge no of java project in simultaneously(assume request comming every time) can i use for sun jdk for this situation. is there way to distributed compilation?

Answer 1

You can use the ToolProvider api to start a compilation programmatically. (see this tutorial for help)

With that as a starting point, you only need to create the proper abstraction to define what a project is (source folders, classpath etc.) and you have a thin frontend for compiling multiple projects.

I would probably use a ThreadPool to implement the actual compiling:

ExecutorService threadPool = 
    Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());

Or if you can't use threads, use a pool of compilers (or compiler holders) with a library like commons / pool.

Either way, you can have this workflow:

1. Request comes in
2. transform request to project definition
3. either request a free compiler or put the project in a queue
4. compile
5. return a handle to the compile job, so the client can poll the status asynchronously

Answer 2

Maven is starting to support it... but why? In the time it takes to do a parallel build of Firefox (a typical, large C++ app) on say... a Core 2 Duo, you probably could build at least 5,000 independent Java apps so long as your build process was properly glued together.

(In the time it takes to build about a half dozen or so large C++ objects for Firefox on my MacBook Pro, I could probably build the entire CAS webapp which has a few hundred Java classes and about 12-18 Maven submodules)