5

I'm looking at restructuring a large Maven project...

A basic overview of our current structure:

build [MVN plugins, third party dependency management]:5.1
   NRW Utils:6.0.0.0-beta12-SNAPSHOT
      server-utils:6.0.0.0-beta12-SNAPSHOT
      ... 
   CMW Root:6.0.0.0-beta12-SNAPSHOT
      cmw-webapp:6.0.0.0-beta12-SNAPSHOT
      cmw-core [dependencies on NRW Utils]:6.0.0.0-beta12-SNAPSHOT
      ...
   NRW Root :6.0.0.0-beta12-SNAPSHOT
      nrw-webapp [depends on NRW Utils & CMW Root modules]:6.0.0.0-beta12-SNAPSHOT
      ...

The reason for change:

The size of each collective module (i.e. NRW Utils, CMW Root and NRW Root) is getting larger and the build process is beginning to take an unbearable amount of time (~4hrs sometimes).

The new plan: 

build [MVN plugins, third party dependency management]:5.1
   NRW Utils:6.0.0.0-NU-beta4-SNAPSHOT
      server-utils:6.0.0.0-NU-beta4-SNAPSHOT
      ... 
   CMW Root:6.0.0.0-CMW-beta12-SNAPSHOT
      cmw-webapp:6.0.0.0-CMW-beta12-SNAPSHOT
      cmw-core [dependencies on NRW Utils]:6.0.0.0-CMW-beta12-SNAPSHOT
      ...
   NRW Root :6.0.0.0-NRW-beta9-SNAPSHOT
      nrw-webapp [depends on NRW Utils & CMW Root modules]:6.0.0.0-NRW-beta9-SNAPSHOT
      ...

We have started introducing 'keys' in the version to distinguish between different 'collective modules' and hence can easily perform stepped releases. In addition, our utility modules are far more stable so we may not require nearly as many beta releases - there is now no restriction on keeping beta numbers in sync.

It is also worth noting that there are in fact 5 different 'collective modules' (not just 3) all to be built with different versions (distinguished by unique keys) which is why I thought it'd be nice to have a centralised place for versions, as opposed to duplicate properties in 5 different POMs.

The problem now lies in the contents of POM files when defining dependencies on modules in a different 'collective module' of a different version.

The proposed solution to dependency version management:

build [MVN plugins, third party dependency management]:5.1
   nrw-version-management:6.0.0.0-beta-SNAPSHOT
      [contains properties defining latest versions of each collective module]   
      NRW Utils:6.0.0.0-NU-beta4-SNAPSHOT
         server-utils:6.0.0.0-NU-beta4-SNAPSHOT
         ... 
      CMW Root:6.0.0.0-CMW-beta12-SNAPSHOT
         cmw-webapp:6.0.0.0-CMW-beta12-SNAPSHOT
         cmw-core [dependencies on NRW Utils]:6.0.0.0-CMW-beta12-SNAPSHOT
         ...
      NRW Root :6.0.0.0-NRW-beta9-SNAPSHOT
         nrw-webapp [depends on NRW Utils & CMW Root modules]:6.0.0.0-NRW-beta9-SNAPSHOT
         ...

nrw-version-management (pom.xml):

...
<parent>
    <groupId>com.project</groupId>
    <artifactId>build</artifactId>
    <version>5.1</version>
</parent>
<groupId>com.project</groupId>
<artifactId>nrw-versions-manager</artifactId>
<version>6.0.0.0-beta-SNAPSHOT</version>
<name>Version Maven Properties</name>
<description>A centralised place for all module property versions</description>
<packaging>pom</packaging>
<properties>
    <nrw.utilities.version>6.0.0.0-NU-beta4-SNAPSHOT</nrw.utilities.version>
    <nrw.cmw.version>6.0.0.0-CMW-beta12-SNAPSHOT</nrw.cmw.version>
    <nrw.version>6.0.0.0-NRW-beta9-SNAPSHOT</nrw.version>
</properties>
...

CMW Root (pom.xml):

...
<parent>
    <groupId>com.project</groupId>
    <artifactId>nrw-versions-manager</artifactId>
    <version>${nrw.core.version}</version>
    ...
</parent>
<groupId>com.project</groupId>
<artifactId>CMW-root</artifactId>
<version>6.0.0.0-CMW-beta12-SNAPSHOT</version>
<packaging>pom</packaging>
<dependencyManagement>
    <dependencies>
        ...
        <dependency>
            <groupId>${project.groupId}</groupId>
            <artifactId>server-utils</artifactId>
            <version>${nrw.utilities.version}</version>
        </dependency>
        ...
</dependencyManagement>
<profiles>
    <profile>
        <id>all</id>
        <modules>
            <module>cmw-webapp</module>
            <module>cmw-core</module>
            ...
        </modules>
    </profile>
    ...
</profiles>
...

N.B. the property ${nrw.core.version} would then be set to 6.3.0.0-beta-SNAPSHOT for a snapshot build via command line arguments (or a default property value).

A possible release process (for 6.0.0.0):

  1. Build the 5.1 build module if not already built
  2. Build nrw-version-management 6.0.0.0 (to avoid snapshot dependencies - however no properties are changed yet)
  3. Build NRW Utils 6.0.0.0-NU cmd args: -Dnrw.core.version=6.0.0.0
  4. Build CMW Root 6.0.0.0-CMW cmd args: -Dnrw.core.version=6.0.0.0 -Dnrw.utilities.version=6.0.0.0-NU
  5. Build NRW Root 6.0.0.0-NRW cmd args: -Dnrw.core.version=6.0.0.0 -Dnrw.utilities.version=6.0.0.0-NU -Dnrw.cmw.version=6.0.0.0-CMW
  6. Re-build nrw-version-management 6.0.0.0 for repository cmd args: -Dnrw.core.version=6.0.0.0 -Dnrw.utilities.version=6.0.0.0-NU -Dnrw.cmw.version=6.0.0.0-CMW
  7. Build nrw-version-management 6.1.0.0-beta-SNAPSHOT with new dev versions and update POM file

The problem:

The build process seems long-winded, especially related to the nrw-version-management module. Also, I started seeing this warning:

'version' contains an expression but should be a constant

And having done some investigating I now understand expressions are not recommended when setting versions (when specifying a parent POM):

The questions:

  • Can I just ignore this warning? Some posts are starting suggest that it might just be acceptable to specify parent POM versions using properties.
  • Is this general approach conventional? Or flawed?
  • Are there better solutions to address the re-structuring of this growing project?

Thanks in advance.

Community
  • 1
  • 1
Ed .
  • 6,373
  • 8
  • 62
  • 82

3 Answers3

2

The benefits of having one single version for all sub modules is simplicity, a benefit that shouldn't be under estimated.

You should indeed ask yourself several times if you really want to depend on different versions of modules in the same hierarchy. As you pointed out release becomes cumbersome.

By handling relationships in a standardized way with single version and deploying to your local repo after successfull builds you should be able to enjoy dependency management the way it was intended.

If you restructure your project so that it adheres to the conventions of the release plugin, release becomes a breeze.

obr
  • 131
  • 7
  • Thanks for your input. The reason that we are veering away from a single version is without that constraint it'll be so much easier to break our build down into 'bitesize' chunks, with the added bonus of simplicity; we now don't need to build everything again if one module has an update. Any more thoughts / ways around that though with a single version? Thanks. – Ed . Jan 20 '12 at 13:56
  • You get the bitesize chunks by sharing a repo for your artifacts. Admittedly you have to build the whole project when you update the version number, but after that you only build and deploy the modules where there has been any development. /oskar – obr Jan 20 '12 at 14:43
  • Hmmm - but we are already using a repository for built artifacts. And leaving the build on a single version still means we have to build the project as one, which leaves us with the time consuming build process (4hrs). Plus our version numbers change frequently (when it is only certain modules that cause that). – Ed . Jan 23 '12 at 17:46
  • But if you begin to use different versions of your submodules, you'll fall in a dependancy management nightmare... – ndeverge Jan 24 '12 at 09:43
2

As nico_ekito stated in one of the comments, in my own opinion, you are falling right now into the "dependency management nightmare". As per your scenario... isn't the fact of managing different versions for submodules (as well as different code stability) a symptom of that those submodules are no longer part of the big project?

I mean, we, as humans, can clearly state that those modules are part of the big picture, the whole approach for solving a problem. That is quite common when you find that your utility classes have no use outside the scope of that big picture. But, from the Maven point of view I think that you are actually dealing with different projects rather than modules of a big one.

So, maybe it's worth to separate the codebases in different structures and just simply reference the utility jars from the modules that actually require it and just use stable builds instead of "last version of that dependency". If working with stable builds it's not feasible because, even though the code for that dependencies is quite stable, it keeps changing at the same time that the code from the other modules, then the solution that Maven gives you is using SNAPSHOTS.

I think you will find some benefits when breaking the code base into separate projects:

  • That way Maven won't trigger a build in those utility modules when it's not needed.
  • If can find a way of just using stable releases for the dependencies you will avoid a lot of hassle.
  • You can also break the team and make the ones to work with just the utility jars do so on a "under-demand basis" from the team that consumes that dependencies.
  • If those utilities are abstract or reusable enough you may find someday another project that may consume them as well without the need of referencing a submodule from a larger project.

There is a lot of discussion in the Internet about the way Maven handles dependencies (and other things) and a lot of people complain about the same problems you are facing. In my own opinion, Maven is a great tool, designed to save us a lot of time when starting new projects but sometimes it can be a bit cumbersome. I started to take a look a other tools like Gradle but now changing the build tool may be worse thing to do. Just giving a though to the idea of breaking the code base, sometimes we can solve a "software problem" just with a bit of "project management" (or "portfolio mangement" in this case).

Alonso Dominguez
  • 7,750
  • 1
  • 27
  • 37
0

I don't feel that sticking to a single version across the whole project is viable at this stage, given that the build process takes so long right now (the other day it failed after 7hrs!!).

However, the two links below don't completely answer my question but are getting there:

Community
  • 1
  • 1
Ed .
  • 6,373
  • 8
  • 62
  • 82