0

I have a multi-module Maven project that installs a whole bunch of artifacts (with different classifiers) into the local Maven repository. I also have a second Maven project that uses the Maven Dependency Plugin to collect those artifacts into a number of different directories (for installer building purposes). And finally I have a Jenkins that I want to do all that for me.

There are a number of requirements I would like to see fulfilled:

  1. Building the source code (and running the tests) and building the installers should be two separate jobs, Job A and Job B.
  2. Job A needs to finish quickly; as it contains the tests the developers should get feedback as fast as possible.
  3. The artifacts of Job B take up a lot of space but they need to be archived so this job should only run when the results of Job A do meet certain requirements (which are not a part of this problem).
  4. Job B needs to be connected to Job A. It must be possible to tell exactly which Job A instance created the files that were used in the build of Job B. (It is also possible that I need a run of Job B for a particular build of Job A which was three weeks and 200 builds ago.)
  5. And finally both jobs should be able to be executed locally on a developer’s machine so I would love to keep most of the configuration within Maven and only relegate to Jenkins what’s absolutely necessary. (Using the Copy Artifacts Plugin I can collect the artifacts from Job A into the required directories in Job B but when removing the collection from the Maven project I also take away the developer’s ability to do local builds.)

Parts of 3 and 4 can be achieved using the Promoted Builds plugin for Jenkins. However, I cannot seem to make sure that the files collected in Job B are exactly the files created by a certain run of Job A. During development all our version numbers of all involved projects are suffixed with “-SNAPSHOT” so that an external job has no way of knowing whether it actually got the correct file or whether it was given a newer file because another instance of Job A has been running concurrently. The version numbers are then increased directly before a release.

Here are some things I have tried and found to be unsatisfactory:

  1. Use a local repository in the workspace directory of Job A. This will, upon each build, download all of the dependencies from our Nexus. While this does not have a huge impact on the diskspace it does consume way too much time.
  2. Merge Job A and Job B into a single job. As Job B takes more time than time A, developers have to wait longer for feedback, it still uses a lot of diskspace—and it doesn’t really solve the problem as there is still the possibility of another Job A+B running at the same time.

Am I missing something obvious here? Are Maven or Jenkins or the combination of both unable to do what I want? What else could I try?

Bombe
  • 81,643
  • 20
  • 123
  • 127
  • First problem: `with different classifiers) into the local Maven repository. ` why not using a repository manager makes also setup and working on Jenkins easier... Why using maven-dependency-plugin to collect into different directories? Do you like to package those deps into tar/zip/archive? Yes use maven-assembly-plugin instead... better solution. About how many tests are we talking about? Second how long does it take to build your project? What do you mean by: "take up a lot of space`? – khmarbaise Apr 11 '22 at 15:27
  • `While this does not have a huge impact on the diskspace it does consume way too much time.` really talking about diskspace? About which sizes we are talking? Unforutnatley I don't know enough detail what you like to and more important why do you like to go that way? – khmarbaise Apr 11 '22 at 15:28
  • The dependencies are collected into different directories for another tool to process them… I have looked into the Maven Assembly Plugin but that looks like a lot more manual work than using the Maven Dependency Plugin. (The other questions about time and diskspace I will ignore because they will inevitably lead to “use a faster/bigger machine” which due to external circumstances is not a valid solution here.) – Bombe Apr 12 '22 at 08:52

0 Answers0