How can I override remote resources when building Maven?

Question

I'm trying to build Maven 3.5.2 from source. _{(Reason: I'm on a machine which has Maven 3.0.5 installed, but is not allowed to download binaries (except for Maven dependencies), and I need a later version of Maven, so I'm planning to use Maven 3.0.5 to build Maven 3.5.2. According to the docs, this is possible.)}

The machine I'm using to build also cannot access the internet.

The Maven build tries to access the Apache license from apache.org and fails if the license file cannot be downloaded. In particular, here is the tail end of the build output (which is running as root, albeit under SElinux):

[INFO] --- apache-rat-plugin:0.11:check (rat-check) @ apache-maven ---
[INFO] 51 implicit excludes (use -debug for more details).
[INFO] Exclude: src/test/resources*/**
[INFO] Exclude: src/test/projects/**
[INFO] Exclude: src/test/remote-repo/**
[INFO] Exclude: **/*.odg
[INFO] Exclude: src/bin/m2.conf
[INFO] Exclude: bootstrap/**
[INFO] Exclude: README.bootstrap.txt
[INFO] Exclude: .repository/**
[INFO] Exclude: .maven/spy.log
[INFO] Exclude: .java-version
[INFO] Exclude: README.md
[INFO] Exclude: DEPENDENCIES
[INFO] 19 resources included (use -debug for more details)
[INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 16 licence.
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:unpack-dependencies (unpack-jansi-native) @ apache-maven ---
[INFO] Unpacking /root/.m2/repository/org/fusesource/jansi/jansi/1.16/jansi-1.16.jar to /tmp/maven-build/apache-maven/target/dependency with includes "META-INF/native/**" and excludes ""
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ apache-maven ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Maven ...................................... SUCCESS [25.091s]
[INFO] Maven Model ....................................... SUCCESS [18.000s]
[INFO] Maven Artifact .................................... SUCCESS [4.418s]
[INFO] Maven Plugin API .................................. SUCCESS [4.677s]
[INFO] Maven Builder Support ............................. SUCCESS [1.900s]
[INFO] Maven Model Builder ............................... SUCCESS [5.690s]
[INFO] Maven Settings .................................... SUCCESS [1.905s]
[INFO] Maven Settings Builder ............................ SUCCESS [2.010s]
[INFO] Maven Repository Metadata Model ................... SUCCESS [1.511s]
[INFO] Maven Artifact Resolver Provider .................. SUCCESS [5.110s]
[INFO] Maven Core ........................................ SUCCESS [13.168s]
[INFO] Maven SLF4J Simple Provider ....................... SUCCESS [5.013s]
[INFO] Maven Embedder .................................... SUCCESS [3.617s]
[INFO] Maven Compat ...................................... SUCCESS [4.462s]
[INFO] Apache Maven Distribution ......................... FAILURE [4:18.467s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5:58.134s
[INFO] Finished at: Mon Feb 12 21:03:11 GMT 2018
[INFO] Final Memory: 83M/190M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on project apache-maven: Error rendering velocity resource. Invocation of method 'getResourceAsFile' in  class org.codehaus.plexus.resource.DefaultResourceManager threw exception org.codehaus.plexus.resource.loader.ResourceNotFoundException: Could not find resource 'https://www.apache.org/licenses/LICENSE-2.0.txt'. at remote-resources[line 38, column 26] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :apache-maven

I think maybe the remote resource plugin is something which is pulled in from the Apache parent POM, but I'm not sure, as I can't quite figure out how this works. That configuration seems to be pulling in a dependency apache-jar-resource-bundle which already contains the LICENSE.txt, so there would be no need to get it from the Internet.

Is there a way to disable the remote resource retrieval (perhaps in favour of a local resource), preferably by passing command-line arguments to Maven, but failing that, by modifying my local copy of Maven's root POM?

EDIT: Adding full stack trace. It seems that the method getResourceAsFile was called using reflection from Velocity, which made me think there must be a Velocity template referencing getResourceAsFile in one of the Apache artifacts, but I couldn't find anything in the repo at https://svn.apache.org/repos/asf/maven/resources/

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on project apache-maven: Error rendering velocity resource. Invocation of method 'getResourceAsFile' in  class org.codehaus.plexus.resource.DefaultResourceManager threw exception org.codehaus.plexus.resource.loader.ResourceNotFoundException: Could not find resource 'https://www.apache.org/licenses/LICENSE-2.0.txt'. at remote-resources[line 38, column 26] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (default) on project apache-maven: Error rendering velocity resource.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:508)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:414)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:357)
Caused by: org.apache.maven.plugin.MojoExecutionException: Error rendering velocity resource.
    at org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.processResourceBundles(ProcessRemoteResourcesMojo.java:1246)
    at org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.execute(ProcessRemoteResourcesMojo.java:520)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
    ... 19 more
Caused by: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getResourceAsFile' in  class org.codehaus.plexus.resource.DefaultResourceManager threw exception org.codehaus.plexus.resource.loader.ResourceNotFoundException: Could not find resource 'https://www.apache.org/licenses/LICENSE-2.0.txt'. at remote-resources[line 38, column 26]
    at org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:243)
    at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:187)
    at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
    at org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:567)
    at org.apache.velocity.runtime.parser.node.ASTExpression.value(ASTExpression.java:71)
    at org.apache.velocity.runtime.parser.node.ASTSetDirective.render(ASTSetDirective.java:142)
    at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
    at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87)
    at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
    at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87)
    at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
    at org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420)
    at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207)
    at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
    at org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420)
    at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207)
    at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
    at org.apache.velocity.runtime.RuntimeInstance.render(RuntimeInstance.java:1378)
    at org.apache.velocity.runtime.RuntimeInstance.evaluate(RuntimeInstance.java:1314)
    at org.apache.velocity.app.Velocity.evaluate(Velocity.java:254)
    at org.apache.maven.plugin.resources.remote.ProcessRemoteResourcesMojo.processResourceBundles(ProcessRemoteResourcesMojo.java:1218)
    ... 22 more
Caused by: org.codehaus.plexus.resource.loader.ResourceNotFoundException: Could not find resource 'https://www.apache.org/licenses/LICENSE-2.0.txt'.
    at org.codehaus.plexus.resource.DefaultResourceManager.getResource(DefaultResourceManager.java:173)
    at org.codehaus.plexus.resource.DefaultResourceManager.getResourceAsFile(DefaultResourceManager.java:91)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:508)
    at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:395)
    at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:384)
    at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:173)
    ... 41 more

EDIT 2: I think I found the culprit: https://github.com/apache/maven/blob/maven-3.5.2/apache-maven/src/main/appended-resources/META-INF/LICENSE.vm

It uses getResourceAsFile with the license URL.

But I don't know where this is pulled in from the remote resources plugin.

I'm not familiar with Apache Velocity. Can anyone help with how to override it?

_{NOTE: Not a duplicate of question 33548395, because that question is about dependency resolution, but this is about remote resources, which is different. The file being downloaded is not a Maven dependency, it's just a text file. The corresponding Maven dependency I would expect it to use is apache-jar-resource-bundle, but that dependency is already present in the remote repository I have access to, so I don't need to force Maven to read it locally. However it does seem that it tries to download the file directly rather than via a Maven dependency.}

Possible duplicate of [How do I force Maven to use my local repository rather than going out to remote repos to retrieve artifacts?](https://stackoverflow.com/questions/33548395/how-do-i-force-maven-to-use-my-local-repository-rather-than-going-out-to-remote) — no id, Feb 12 '18 at 22:19
@IliaNedoluzhko I don't think it's a duplicate. That question is about Maven dependency resolution, but my dependency resolution is working fine; furthermore, the artifact `apache-jar-resource-bundle` is present in the remote repository and if all it did was pull it from the remote, I wouldn't have any problem. This is about it trying to pull a file directly from `apache.org`, i.e. not from a Maven repository. — Adam Burley, Feb 12 '18 at 22:31
according to the plugin docs there is a skip property: http://maven.apache.org/plugins/maven-remote-resources-plugin/process-mojo.html#skip - so adding something like "-Dremoteresources.skip=true" to the goal you execute might do - depending whats happening later on with the file. Otherwise you need to fake the url by setting up a local server and adding the site to your hosts file if changing the build files itself turns out to be too difficult. — wemu, Feb 12 '18 at 23:28
@wemu thanks, that works, although it's not very ideal. What that basically means is I have built an Apache binary without including the licenses. I'm not even sure if that is legal. it would be better if there was some way to override the URL it is looking for. and I still don't even really understand **why** that plugin is trying to access an external URL (not dependency) — Adam Burley, Feb 13 '18 at 00:20
well a "mvn help:effective-pom" only lists that license url in the license section of the pom, not a property or plugin config. So it might just makes sure the legal files are where they belong to. There is some "patching" documentation: http://maven.apache.org/plugins/maven-remote-resources-plugin/supplemental-models.html - but all this feels like a waste of time at this point. Using open source software for your builds and then restrict yourself (the company) from it... someone will be able to download a zip. Find that someone. — wemu, Feb 13 '18 at 07:42
@wemu I'm sure someone can download a ZIP and put it onto the machine, but that would be a manual process, and the build must be automated. Bottom line is I need to find a way to resolve this problem. Thanks for checking the effective POM. Also it seems that this error came from Apache Velocity. I can see that https://svn.apache.org/repos/asf/maven/resources/tags/apache-jar-resource-bundle-1.4/src/main/resources/META-INF/DEPENDENCIES.vm does reference the license URL, but I'm not sure why it would actually try to access the URL as opposed to just write it out into the template... — Adam Burley, Feb 13 '18 at 11:26
well at some point some infrastructure must be there. not sure why you would need to automate the build step. so more ideas: use the official maven docker images: https://store.docker.com/images/maven - if you can run docker images. or try the https://github.com/takari/maven-wrapper (similar to the gradle wrapper). A simple script that will download the distribution. That might work through your company proxy (I think the distributions are also in maven central) — wemu, Feb 13 '18 at 19:42
no can't run docker images, also even if I could I would still need a way to get the (binary) docker image onto the machine. and I had a look at maven wrapper, it retrieves Maven by looking for e.g. `/org/apache/maven/apache-maven/3.2.1/apache-maven-3.2.1-bin.zip` within Maven Central, but this file does not exist in our internal repository - it's only for JARs, not ZIPs. Bottom line is I need to resolve the problem specified in my question which is about building Maven. I have added another edit yesterday as I think I found the file within the source codebase that is causing the problems. — Adam Burley, Feb 14 '18 at 13:55

Adam Burley · Accepted Answer · 2018-02-14T22:16:08.350

After realising that LICENSE.vm (and the one inside apache-maven, not the other one) was the problem (see my "EDIT 2"), I found that it also contains the solution. A URL for Glassfish is manually redirected to another and this is in the file at lines 32-35. So I thought I can do something similar for my scenario.

First I worked out all the licenses and downloaded them. I gave them all .license extensions so I could tell Git to treat files with this extension as binary (via .gitattributes) so it wouldn't try to mess with the files by converting line endings, as I wanted to replicate what the actual build would do. There were only 5 files I actually needed to download in all.

Secondly I created a little snippet to insert into the VM file; I called this license-redirect-snippet.vm:

## redirect all license URLs to local files
#if ($url == "https://www.apache.org/licenses/LICENSE-2.0.txt")
#set ( $url = '/usr/local/share/maven-build-licenses/apache2-txt.license' )
#end
#if ($url == "http://www.apache.org/licenses/LICENSE-2.0.txt")
#set ( $url = '/usr/local/share/maven-build-licenses/apache2-txt.license' )
#end
#if ($url == "http://www.apache.org/licenses/LICENSE-2.0")
#set ( $url = '/usr/local/share/maven-build-licenses/apache2-html.license' )
#end
#if ($url == "https://glassfish.java.net/public/CDDLv1.0.html")
#set ( $url = '/usr/local/share/maven-build-licenses/cddl1-html.license' )
#end
#if ($url == "http://www.eclipse.org/legal/epl-v10.html")
#set ( $url = '/usr/local/share/maven-build-licenses/epl1-html.license' )
#end
#if ($url == "http://www.opensource.org/licenses/mit-license.php")
#set ( $url = '/usr/local/share/maven-build-licenses/mit-html.license' )
#end

NB: there is a new line at the end of the file.

Then it was just a question of copying my licenses into the given folder (/usr/local/share/maven-build-licenses/) and using sed to modify the VM file before build, adding the snippet after line 35:

sed -i '35r /tmp/license-redirect-snippet.vm' /tmp/maven-build/apache-maven/src/main/appended-resources/META-INF/LICENSE.vm

After that, the build passes!!

I probably could have come up with something better e.g. dynamically generating the snippet file based on the contents of the directory and some key-value pairs, or even rewriting the getResourceAsFile method so that this variable rewriting would not be needed, but that seems like overkill for such a simple case where there are only 5 licenses to deal with.

How can I override remote resources when building Maven?

1 Answers1