How do you resolve dependencies across all modules in a maven plugin?

Question

I'm writing a Maven plugin that gets the resolved dependencies. It works fine for a single module project/pom, but fails on multiple module projects.

Here's a code snippet

@Mojo(
  name="scan",
  aggregator = true,
  defaultPhase = LifecyclePhase.COMPILE,
  threadSafe = true,
  requiresDependencyCollection = ResolutionScope.TEST,
  requiresDependencyResolution = ResolutionScope.TEST,
  requiresOnline = true
)
public class MyMojo extends AbstractMojo {

  @Parameter(property = "project", required = true, readonly = true)
  private MavenProject project;

  @Parameter(property = "reactorProjects", required = true, readonly = true)
  private List<MavenProject> reactorProjects;


  @Override
  public void execute() throws MojoExecutionException {
    for(MavenProject p : reactorProjects) {
      for(Artifact a : p.getArtifacts()) {
         ...consolidate artifacts
      }
    }
  }
}

The above code will consolidate all the resolved dependencies across all the modules, but it includes some additional ones.

Here's a sample project to work with. Please download this github repo

From the modules-project main folder, please run

mvn dependency:tree -Dverbose -Dincludes=commons-logging

You should see an output like this

[INFO] ------------------------------------------------------------------------
[INFO] Building core 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ core ---
[INFO] com.github:core:jar:0.1-SNAPSHOT
[INFO] \- axis:axis:jar:1.4:compile
[INFO]    +- commons-logging:commons-logging:jar:1.0.4:runtime
[INFO]    \- commons-discovery:commons-discovery:jar:0.2:runtime
[INFO]       \- (commons-logging:commons-logging:jar:1.0.3:runtime - omitted for conflict with 1.0.4)
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building web 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ web ---
[INFO] com.github:web:war:0.1-SNAPSHOT
[INFO] +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] \- com.github:core:jar:0.1-SNAPSHOT:compile
[INFO]    \- axis:axis:jar:1.4:compile
[INFO]       +- (commons-logging:commons-logging:jar:1.0.4:runtime - omitted for conflict with 1.1.1)
[INFO]       \- commons-discovery:commons-discovery:jar:0.2:runtime
[INFO]          \- (commons-logging:commons-logging:jar:1.0.3:runtime - omitted for conflict with 1.1.1)
[INFO] ------------------------------------------------------------------------

Notice that the module/project core depends on commons-logging 1.0.4 and commons-logging 1.0.3, but 1.0.3 is omitted due to a conflict and 1.0.4 is resolved. This means that if you were to build core on its own, you should only get commons-logging 1.0.4.

Notice that module/project web depends on conflicting versions of commons-logging as well but resolves to 1.1.1.

Now if you were to build the "entire project" (modules-project) with the "mvn package" command, you should see that modules-project/web/target/myweb/WEB-INF/lib contains all the resolved dependencies and it includes ONLY commons-logging 1.1.1.

Here's the problem with the code

In the above code, reactorProjects is instantiated with 3 MavenProject's: modules-project, core, and web.

For modules-project and web, it resolves and returns commons-logging 1.1.1. However, for the core project, it resolves and returns commons-logging 1.0.4.

I want my plugin code to know that commons-logging 1.1.1 is the dependency that the build will produce, and not commons-logging 1.0.4

Any thoughts?

What is the purpose of this plugin? What would you like to achive? — khmarbaise, Apr 13 '16 at 07:24
I just want to get all the dependencies that "mvn package" would produce — kane, Apr 13 '16 at 14:59
You should use the maven-assembly-plugin which already can do such things...Apart from that that you like to get the dependencies was clear based on the question but what's not clear is why do you need them? — khmarbaise, Apr 13 '16 at 16:14
The why is for a company project. It has to do with security and that's all I can really say unfortunately. Can you elaborate on what you mean by "use the maven-assembly-plugin"? Could you give me a snippet of code or point me to where I can find java code? Thanks — kane, Apr 13 '16 at 18:15
IntelliJ shows me the whole dependency tree graphically. Is that what you mean? — duffymo, Apr 18 '16 at 21:42
There's something I don't quite get. When you're packaging a project, you're packaging a single Maven Project, that has some dependencies. The dependencies that will be resolved for the packaged artifact are thus the dependencies that are resolved for that particular Maven project. It does not depend of the dependencies of all the other modules in the reactor. Your question is unclear to me. Yes, those dependencies can be other modules in the reactor, but they remain dependencies of the single packaged Maven project. — Tunaki, Apr 18 '16 at 21:49
@duffymo Yes, I'd like to get the dependency tree for multi-module projects within a maven plugin implementation — kane, Apr 18 '16 at 21:51
Are you interested in something like this? http://stackoverflow.com/a/35380442/1743880 — Tunaki, Apr 18 '16 at 21:53
@Tunaki Within a plugin implementation (ie implementing AbstractMojo) I can only get resolved dependencies within a module. I don't know how to get resolved dependencies for an entire project with multiple modules. Please let me know if my toy example is clear enough and how I can make that clearer — kane, Apr 18 '16 at 22:04
I think you are confused about what a project is. A Maven project is simply the `pom.xml` and the corresponding sources inside `src` (or not, there may not be any sources). A project with `pom` packaging is a Maven project. A module of a multi-module project is a Maven project. There is no "entire project". There might be a packaging specific project that only purpose is to package, but it is a Maven project as well. This project will have as dependency other modules (which are other Maven projects). But what will be finally packaged are the dependencies resolved on _that_ particular project. — Tunaki, Apr 18 '16 at 22:08

Tunaki · Accepted Answer · 2016-04-20T18:13:19.653

You practically have all it takes in your question. The following plugin will print in the console output the artifacts of the WAR project in the reactor:

@Mojo(name = "foo", aggregator = true, requiresDependencyResolution = ResolutionScope.TEST)
public class MyMojo extends AbstractMojo {

    @Parameter(defaultValue = "${project}", readonly = true, required = true)
    private MavenProject project;

    @Parameter(defaultValue = "${session}", readonly = true, required = true)
    private MavenSession session;

    @Parameter(property = "reactorProjects", required = true, readonly = true)
    private List<MavenProject> reactorProjects;

    public void execute() throws MojoExecutionException, MojoFailureException {
        MavenProject packagedProject = getWarProject(reactorProjects);
        for (Artifact artifact : packagedProject.getArtifacts()) {
            getLog().info(artifact.toString());
        }
    }

    private MavenProject getWarProject(List<MavenProject> list) throws MojoExecutionException {
        for (MavenProject project : list) {
            if ("war".equals(project.getPackaging())) {
                return project;
            }
        }
        throw new MojoExecutionException("No WAR project found in the reactor");
    }

}

What this does is that it acquires all the projects in the reactor with the injected parameter reactorProjects. Then, it loops to find which one of those is the "war" by comparing their packaging. When it is found, getArtifacts() will return all the resolved artifacts for that project.

The magic that makes it work is the aggregator = true in the MOJO definition:

Flags this Mojo to run it in a multi module way, i.e. aggregate the build with the set of projects listed as modules.

When added to core POM

<plugin>
  <groupId>sample.plugin</groupId>
  <artifactId>test-maven-plugin</artifactId>
  <version>1.0.0</version>
  <executions>
    <execution>
      <id>test</id>
      <phase>compile</phase>
      <goals>
        <goal>foo</goal>
      </goals>
    </execution>
  </executions>
</plugin>

and run with your example project, this prints in the console:

[INFO] commons-logging:commons-logging:jar:1.1.1:compile
[INFO] com.github:core:jar:0.1-SNAPSHOT:compile
[INFO] axis:axis:jar:1.4:compile
[INFO] org.apache.axis:axis-jaxrpc:jar:1.4:compile
[INFO] org.apache.axis:axis-saaj:jar:1.4:compile
[INFO] axis:axis-wsdl4j:jar:1.5.1:runtime
[INFO] commons-discovery:commons-discovery:jar:0.2:runtime

This is good enough. With that, we can go forward and, for example, compare the resolved artifacts by the current project being build and the packaged project. If we add a method

private void printConflictingArtifacts(Set<Artifact> packaged, Set<Artifact> current) {
    for (Artifact a1 : current) {
        for (Artifact a2 : packaged) {
            if (a1.getGroupId().equals(a2.getGroupId()) && 
                    a1.getArtifactId().equals(a2.getArtifactId()) &&
                    !a1.getVersion().equals(a2.getVersion())) {
                getLog().warn("Conflicting dependency: " + a2 + " will be packaged and found " + a1);
            }
        }
    }
}

called with

printConflictingArtifacts(packagedProject.getArtifacts(), project.getArtifacts());

that compares the current artifacts with the artifacts of the packaged project, and only retain those with the same group/artifact id but different version, we can get in the console output with your example:

[WARNING] Conflicting dependency: commons-logging:commons-logging:jar:1.1.1:compile will be packaged and found commons-logging:commons-logging:jar:1.0.4:runtime

The above assumed that our final packaging module was a WAR module. We could make that more generic and let the user specify which one of the module is the target module (i.e. that will package the real delivery).

For that, we can add a parameter to our MOJO

@Parameter(property = "packagingArtifact")
private String packagingArtifact;

This parameter will be of the form groupId:artifactId and will represent the coordinates of the target module. We can then add a method getPackagingProject whose goal will be to return the MavenProject associated with those coordinates.

The configuration of the plugin inside core would be

<plugin>
    <groupId>sample.plugin</groupId>
    <artifactId>test-maven-plugin</artifactId>
    <version>1.0.0</version>
    <executions>
        <execution>
            <id>test</id>
            <phase>compile</phase>
            <goals>
                <goal>foo</goal>
            </goals>
            <configuration>
                <packagingArtifact>com.github:web</packagingArtifact>
            </configuration>
        </execution>
    </executions>
</plugin>

And the full MOJO would be:

@Mojo(name = "foo", aggregator = true, requiresDependencyResolution = ResolutionScope.TEST, defaultPhase = LifecyclePhase.COMPILE)
public class MyMojo extends AbstractMojo {

    @Parameter(defaultValue = "${project}", readonly = true, required = true)
    private MavenProject project;

    @Parameter(defaultValue = "${session}", readonly = true, required = true)
    private MavenSession session;

    @Parameter(property = "reactorProjects", required = true, readonly = true)
    private List<MavenProject> reactorProjects;

    @Parameter(property = "packagingArtifact")
    private String packagingArtifact;

    public void execute() throws MojoExecutionException, MojoFailureException {
        MavenProject packagedProject = getPackagingProject(reactorProjects, packagingArtifact);
        printConflictingArtifacts(packagedProject.getArtifacts(), project.getArtifacts());
    }

    private void printConflictingArtifacts(Set<Artifact> packaged, Set<Artifact> current) {
        for (Artifact a1 : current) {
            for (Artifact a2 : packaged) {
                if (a1.getGroupId().equals(a2.getGroupId()) && a1.getArtifactId().equals(a2.getArtifactId())
                        && !a1.getVersion().equals(a2.getVersion())) {
                    getLog().warn("Conflicting dependency: " + a2 + " will be packaged and found " + a1);
                }
            }
        }
    }

    private MavenProject getPackagingProject(List<MavenProject> list, String artifact) throws MojoExecutionException {
        if (artifact == null) {
            return getWarProject(list);
        }
        String[] tokens = artifact.split(":");
        for (MavenProject project : list) {
            if (project.getGroupId().equals(tokens[0]) && project.getArtifactId().equals(tokens[1])) {
                return project;
            }
        }
        throw new MojoExecutionException("No " + artifact + " project found in the reactor");
    }

    private MavenProject getWarProject(List<MavenProject> list) throws MojoExecutionException {
        for (MavenProject project : list) {
            if ("war".equals(project.getPackaging())) {
                return project;
            }
        }
        throw new MojoExecutionException("No WAR project found in the reactor");
    }

}

This implements the idea of above: when the user has given a target module, we use it as reference. When this parameter is not present, we default to finding a WAR in the reactor.

This is an ingenious solution and I believe it will work for the specific example I listed. (I upvoted it) My question is how general is this solution. e.g. Do all multi-module projects include exactly one war? I assume that if there is no war-project, then this will not work but I'm not sure what a multi-module project without a war packaging means... — kane, Apr 20 '16 at 17:49
@kane No, multi-module projects don't necessarily have one war. The question comes down to determining which of the module is the "packaging" or "target" module. In this case, it is determined with the module having `war` packaging. However, it is possible to make it more generic and adding a configuration element specifying the target module. I'll edit with a solution with that also. — Tunaki, Apr 20 '16 at 17:53
I was reading about the reactor. Apparently, it guarantees the modules are built in order according to dependency. I wonder if there is a general solution where we take advantage of the reactorProjects order. ie will the last project always contain the full set of resolved dependencies; is that our "target" module? — kane, Apr 20 '16 at 17:56
@kane Not necessarily: we could have an unrelated module inside the reactor that has no other dependency on other modules. It could be built as last. Having a configuration element for that seems the best fit. I'm testing something right now. — Tunaki, Apr 20 '16 at 18:02
Good point about having an unrelated module inside the reactor that has no other dependency on other modules. But then wouldn't the war and user-configuration approach missed this module as well if it's unrelated? I do think your user-configuration approach is an improvement — kane, Apr 20 '16 at 19:00
@kane No it wouldn't. It is still a module of the reactor, even if it is unrelated to the rest of the other modules. — Tunaki, Apr 20 '16 at 19:01
I was just thinking, what if I create a project "on-the-fly" and have it depend on all the modules/projects. Could I then have maven give me this on-the-fly project's resolved dependencies...? — kane, Apr 20 '16 at 19:01
@kane This _could_ be possible but it would be waaaay harder and I'm not even sure it would answer your intial question: the difference of resolved dependencies in your case is that `war` has a dependency D but it depends on `core` has a conflicting dependency. If you would create in-memory a 3rd project that depends on `war` and `core`, the sets of resolved dependencies of that in-memory project would be different that the sets of dependencies of `war`. — Tunaki, Apr 20 '16 at 19:05
Thanks for all your help @Tunaki ! I appreciate you walking through all the considerations of the problem and solutions. That's above and beyond answering my question. — kane, Apr 21 '16 at 00:41
I ran into another interesting problem that I'm hoping an Expert Maven developer such as yourself can help answer http://stackoverflow.com/questions/36757645/how-to-get-relocated-or-resolved-artifact-id-from-top-level-dependencies-in-ma — kane, Apr 21 '16 at 00:42

How do you resolve dependencies across all modules in a maven plugin?

1 Answers1