2

I have a multi-module project in which one of the two submodules depends upon the other. This question implies that mvn install is supposed to take care of these dependencies.

Everything is working fine at a local level on my machine (both inside Eclipse and on the command line). However, when I update the module which changes code the second module needs and commit to my github repository, the second module fails the Travis CI build. I've traced the problem down to the fact that it is downloading its dependency from a repository rather than using the one it just compiled.

The problem is very similar to the one reported in this question, except that instead of my local repository, I'm dealing with the Travis CI automated build. The comments and answer on that question indicate that:

The dependency has a snapshot version. For snapshots, Maven will check the local repository and if the artifact found in the local repository is too old, it will attempt to find an updated one in the remote repositories.

However, in this case, "too old" shouldn't apply. The local artifact was literally just built. It should be seconds old.

Here are the relevant sections of pom.xml of the oshi-json module, noting its dependence on oshi-core (both share their version number with the parent):

<parent>
    <groupId>com.github.dblock</groupId>
    <artifactId>oshi-parent</artifactId>
    <version>3.0-SNAPSHOT</version>
</parent>

<artifactId>oshi-json</artifactId>
<packaging>jar</packaging>

<name>oshi-json</name>

<dependencies>
    <dependency>
        <groupId>${project.groupId}</groupId>
        <artifactId>oshi-core</artifactId>
        <version>${project.version}</version>
    </dependency>
</dependencies>

And an excerpt from the parent pom.xml:

<groupId>com.github.dblock</groupId>
<artifactId>oshi-parent</artifactId>
<version>3.0-SNAPSHOT</version>
<packaging>pom</packaging>

<name>oshi-parent</name>

<modules>
    <module>oshi-core</module>
    <module>oshi-json</module>
</modules>

The complete files are available here:

Relevant excerpts from the log files are shown below. A complete example of a failed Travis CI build can be found here. Note that the symptom (Compilation Error) is a result of Travis using the latest snapshot for oshi-core, which does not include the class referenced in oshi-json; however, the class is in the submitted pull request and it compiles (and builds with mvn install) locally.

When I make a new commit, Travis executes the build as follows:

$ mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V

The reactor properly arranges the builds in order:

[INFO] Reactor Build Order:
[INFO] 
[INFO] oshi-parent
[INFO] oshi-core
[INFO] oshi-json

The parent builds, and then oshi-core builds, storing the compiled code locally:

[INFO] Building jar: /home/travis/build/dblock/oshi/oshi-core/target/oshi-core-3.0-SNAPSHOT.jar

This is the jar that I want the next module dependent on. However, when the oshi-json build starts, it instead downloads the artifact from maven instead:

[INFO] Downloading: https://oss.sonatype.org/content/repositories/snapshots/com/github/dblock/oshi-core/3.0-SNAPSHOT/oshi-core-3.0-20160624.032041-9.jar

[INFO] Downloaded: https://oss.sonatype.org/content/repositories/snapshots/com/github/dblock/oshi-core/3.0-SNAPSHOT/oshi-core-3.0-20160624.032041-9.jar (199 KB at 708.5 KB/sec)

EDIT: Just noticed the following warning which occurs just prior to the download, which may or may not be relevant (an error in Travis settings) however, fixing that warning still doesn't resolve my problem, here's a failed build without the warning:

[WARNING] Failure to transfer com.github.dblock:oshi-core:3.0-SNAPSHOT/maven-metadata.xml from https://nexus.codehaus.org/snapshots/ was cached in the local repository, resolution will not be reattempted until the update interval of codehaus-snapshots has elapsed or updates are forced. Original error: Could not transfer metadata com.github.dblock:oshi-core:3.0-SNAPSHOT/maven-metadata.xml from/to codehaus-snapshots (https://nexus.codehaus.org/snapshots/): nexus.codehaus.org

This is a temporary problem associated only with Travis-CI; the pom setup will work fine once I actually release all the modules, and I can work around it by locally executing mvn clean deploy to push the new oshi-core to the OSS repository so Travis is happy. However, this seems a poor workaround.

Is there a way I can tell Travis CI to use the jar that it just compiled rather than downloading a new one -- and to do so in a way that doesn't break the intended download of the dependency from the repository once it's published?

Community
  • 1
  • 1
Daniel Widdis
  • 8,424
  • 13
  • 41
  • 63
  • Can you give from all relevant modules the pom files otherwise it's hard to say something useful. during a multi module build it should not download anything from anywhere cause it will be solved from the reactor. – khmarbaise Jun 25 '16 at 06:17
  • @khmarbaise I expanded the excerpts and linked to the original poms (and a failed build) on my github site. You can probably [fork the upstream project](https://github.com/dblock/oshi) to experiment locally if you like. – Daniel Widdis Jun 25 '16 at 17:36

1 Answers1

2

So Ok I have taken a deeper look into your build let me summarize some of my thoughts here:

I found that you have configured maven-clean-plugin with different values which are not necessary nor do they have an advantage. Go with Convention over Configuration paradigm. Only violate it if you have really good reason (To be honest I don't see one. May be you can explain it).

You have configured the maven-source-plugin to run during the default life cycle which means it will run with every mvn clean package or with mvn install which is usually not necessary, cause source packages are usually only needed if you do a release and transfer to Maven Central. Apart from that you have configured to use the jar goal of maven-source-plugin which forkes the life cycle which means to start some parts from the beginning. The best to remove the forked life cycle goal. Best is to add source packages only during the release run. (Please be aware that you have turned off to use release profile).

So come to maven-javadoc-plugin. You have bound it so that it will run every time via mvn clean package or mvn install. I would recommend to run that only during the site life cycle` cause generating javadoc tooks much time...

For what reason do you have configuration wagon-ssh as extensions? What is the purpose of that? If it's to publish the generated site to github better take at Maven SCM Publish Plugin is much more simpler and faster.

You have also configured maven-assembly-plugin with some strange properties src.relative.loc. If you really have a single descriptor which can be reused you should take a look into the documtation about shared descriptors. Declaring maven-assembly-plugin into the parent means it will be executed for all pom's including the parent which produces the following warnings in your build:

[INFO] --- maven-assembly-plugin:2.6:assembly (oshi-assembly) @ oshi-parent ---
[WARNING] Cannot include project artifact: com.github.dblock:oshi-parent:pom:3.0-SNAPSHOT; it doesn't have an associated file or directory.
[INFO] Building tar: /Users/kama/ws-git-so/oshi/target/oshi-parent-3.0-SNAPSHOT-oshi-assembly.tar.gz
[WARNING] Cannot include project artifact: com.github.dblock:oshi-parent:pom:3.0-SNAPSHOT; it doesn't have an associated file or directory.
[INFO] Building tar: /Users/kama/ws-git-so/oshi/target/oshi-parent-3.0-SNAPSHOT-oshi-assembly.tar.bz2
[WARNING] Cannot include project artifact: com.github.dblock:oshi-parent:pom:3.0-SNAPSHOT; it doesn't have an associated file or directory.
[INFO] Building zip: /Users/kama/ws-git-so/oshi/target/oshi-parent-3.0-SNAPSHOT-oshi-assembly.zip
[INFO]

Ah...very important: you are using the goal assembly which is deprecated and should not be used. Use the only existing one: single nothing else. Cause assembly is also forking the lifecycle which slows down your build.

I'm not sure what you like to produce here? A binary distribution package? If so you should create a separate module which is called something like oshi-dist which contains the configuration of maven-assembly-plugin and may be other distribution related informations like scripts etc. And than the creation should be handled like this. This is related to separation of concerns. Why should the generated site part of this?

BTW: Every plugin which is defined in <build><plugins>... will be inherited to all childs and will be executed which is usually not the right way. The executed plugins should be defined by the appropriate packaging type which is given in the pom.

Update: Why have you configured maven-scm-plugin ?

Update 2: I have created a PR ..so you can take a deeper look into my suggestions... May be i missed some parts of your needs..

khmarbaise
  • 92,914
  • 28
  • 189
  • 235
  • 1
    Thanks for the detailed comments and PR. Most of the pom was borrowed from another project and configured for a single project build about a year ago; some of the problems you note are a result of me just trying to split things up and get the multi module build working. I'll see if these changes fix the local vs. repository issue. – Daniel Widdis Jun 26 '16 at 21:52
  • It looks like the new pom setup fixed my issue, thanks! Didn't do a step by step to see exactly which of the many errors caused it (probably the maven source plugin setup). I'll keep working on refining the pom per your suggestions here, but the main issue is solved. – Daniel Widdis Jun 27 '16 at 05:37
  • @khmarbaise I owe you a beer – German Attanasio Sep 29 '16 at 14:08