Run mvn as a tool in a bazel build

Question

I have a tricky module in my project which cannot be (yet) built by anything but Maven. The good thing is that this module does not have any Java dependencies in my project, and it only produces a .war file that I need to move to the correct place in the output zip distribution.

I want to migrate the build to bazel, but I want to tackle the tricky module in a later stage of the migration.

How can I run mvn as a tool in my bazel build, to let it produce the .war file?

Maven expects to be executed from a directory containing the pom.xml
Maven writes output to a subdirectory of the input filetree
Maven is happiest when settings.xml contains absolute paths (to local maven repository for the .war, stored in git)
Maven uses $HOME/.m2 for caching

I looked at rules_foreign_cc, which creates a wrapper script around cmake. Is this the way to go? https://github.com/bazelbuild/rules_foreign_cc/blob/8372f383cf7277a88762efe25d8cfee10ad27929/tools/build_defs/framework.bzl#L156

Just curious (don't know bazel, but maven): Which of the bullet points seem problematic and why? — J Fabian Meier, Apr 30 '20 at 08:03
@JFMeier All of those ;) Possibly have a look at blog Running tools under Bazel (https://medium.com/@Jakeherringbone/running-tools-under-bazel-8aa416e7090c) for more elaborated bullet points ;) — user7610, Apr 30 '20 at 08:21
Maybe one extra random bullet point, which is not relevant for this specific question, but appeared in general mvn to bazel migration. Maven runs tests from a directory with .class files. Bazel always builds a test jar and runs that. https://stackoverflow.com/questions/43709701/add-custom-folders-to-classpath-in-bazel-java-tests — user7610, Apr 30 '20 at 08:27

score 3 · Answer 1 · answered Apr 30 '20 at 09:35

I have a mvn_package rule prototype in this pull request. The points you raised are crucial; it takes a bit to tame Maven's conventions and expectations in Bazel's stricter action execution environment.

Maven expects to be executed from a directory containing the pom.xml

You can use the -f flag to specify the location of the pom.xml.

e.g. mvn package -f %s -DskipTests -DbazelOutputDir=%s -Pbazel" % (ctx.file.pom_xml.dirname,output_jar.dirname)

Maven writes output to a subdirectory of the input filetree

I created a custom profile in the pom.xml to for a custom Maven define, and passed that define the above command line:

    <profiles>
        <profile>
            <id>bazel</id>
            <build>
                <directory>${bazelOutputDir}</directory>
            </build>
        </profile>
    </profiles>

Maven is happiest when settings.xml contains absolute paths (to local maven repository for the .war, stored in git)

This is unfortunate, and you'd have to find a way around it or disable sandboxing :(

Maven uses $HOME/.m2 for caching

As long as the Bazel action inputs and outputs are declared correctly in the Starlark rule, you should not need to interact with Maven's .m2 cache directory since Bazel has its own action cache.

For completeness, here's the proof of concept of a Starlark rule to package a .jar, so you'll need to maybe tweak it for a .war:

def _mvn_package_impl(ctx):
    inputs = []
    inputs.extend(ctx.files.srcs)
    inputs.append(ctx.file.pom_xml)

    output_jar = ctx.actions.declare_file("target/%s-%s.jar" % (ctx.attr.artifact_id, ctx.attr.version))
    target_dir = ctx.actions.declare_directory("target")
    outputs = [output_jar, target_dir]

    # -Djar.finalName=custom-jar-name
    ctx.actions.run_shell(
        inputs = inputs,
        outputs = outputs,
        mnemonic = "MvnPackage",
        progress_message = "Running 'mvn package' for %s" % output_jar.short_path,
        command = "mvn package -f %s -DskipTests -DbazelOutputDir=%s -Pbazel" % (ctx.file.pom_xml.dirname,output_jar.dirname),
    )

    return [
        DefaultInfo(
            files = depset(outputs),
        ),
        JavaInfo(
            output_jar = output_jar,
            compile_jar = output_jar,
        ),
    ]

mvn_package = rule(
    implementation = _mvn_package_impl,
    attrs = {
        "srcs": attr.label_list(allow_files = True, allow_empty = False),
        "pom_xml": attr.label(allow_single_file = True, mandatory = True),
        "artifact_id": attr.string(mandatory = True),
        "group_id": attr.string(mandatory = True),
        "version": attr.string(mandatory = True),
    },
)

and using the above rule in a BUILD file:

load("@rules_jvm_external//:mvn.bzl", "mvn_package")

mvn_package(
    name = "hello_maven",
    srcs = [
        "//src/main/java/hello:srcs",
    ],
    pom_xml = "//:pom.xml",
    group_id = "org.springframework",
    artifact_id = "gs-spring-boot",
    version = "0.1.0",
)

java_binary(
    name = "hello_maven_app",
    runtime_deps = [":hello_maven"],
    main_class = "hello.Application",
)

$HOME/.m2/repository should not be accessed concurrently (https://issues.apache.org/jira/browse/MNG-2802), but it often is, as bazel likes to run multiple actions by default, possibly multiple mvn rules at the same time. Absolute paths in `settings.xml` actually turned out to be easy. In run_shell, I can set env variable (I am calling it ACTIONROOT, and then in settings.xml I set repo url to `file://${env.ACTIONROOT}/...` — user7610, May 01 '20 at 09:55
The "processwrapper-sandbox" is happy letting maven interact with $HOME/.m2, but the "linux-sandbox" blocks all writes there. I was using processwrapper so far, but I want to switch to linux-sandbox, since it is the default on Linux (if available) anyways. — user7610, May 01 '20 at 14:08
I had to add additional attributes to this rule. I needed to specify my own `-s settings.xml`, for example. I also found out that I need to run other maven phases, besides `package`, and that I sometimes do want to build a project with subprojects, and access `some/subproject/target` as ouptut, and not just the top level `target`. — user7610, May 19 '20 at 08:47

Run mvn as a tool in a bazel build

1 Answers1