maven-surefire-report-plugin not picking up its configuration

Question

I've got a problem with the maven-surefire-report-plugin. I've added some reportSet configurations in the reporting section of my pom.xml, and it doesn't seem to pick up this configuration.

I was lead to understand that the reporting section was the way to go for such plugins, but I'm beginning to doubt so.

Here is my pom.xml's reporting section:

<reporting>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-report-plugin</artifactId>
            <version>2.22.2</version>

            <configuration>
                <linkXRef>false</linkXRef>
            </configuration>

            <reportSets>
                <reportSet>
                    <id>aggregated-unit-test-report</id>

                    <configuration>
                        <outputDirectory>${project.reporting.outputDirectory}/test-reports/ut</outputDirectory>
                    </configuration>

                    <reports>
                        <report>report-only</report>
                    </reports>
                </reportSet>
            </reportSets>
        </plugin>
    </plugins>
</reporting>

And here is (partly) what I got from running mvn surefire-report:report-only -X on that module:

// Cropped for brevity
[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-surefire-report-plugin:3.0.0-M5:report-only' with basic configurator -->
[DEBUG]   (f) aggregate = false
[DEBUG]   (f) alwaysGenerateSurefireReport = true
[DEBUG]   (f) inputEncoding = ISO-8859-1
[DEBUG]   (f) linkXRef = true
[DEBUG]   (f) outputDirectory = C:\Users\francois.dupire\workspace\forhrm\libs\framework\coverage\target\site
[DEBUG]   (f) outputName = surefire-report
[DEBUG]   (f) project = MavenProject: be.formatech.forhrm.framework:framework-coverage:1.0.0-SNAPSHOT @ C:\Users\francois.dupire\workspace\forhrm\libs\framework\coverage\pom.xml
[DEBUG]   (f) reactorProjects = [MavenProject: be.formatech.forhrm.framework:framework-coverage:1.0.0-SNAPSHOT @ C:\Users\francois.dupire\workspace\forhrm\libs\framework\coverage\pom.xml]
[DEBUG]   (f) showSuccess = true
[DEBUG]   (f) skipSurefireReport = false
[DEBUG]   (f) xrefLocation = C:\Users\francois.dupire\workspace\forhrm\libs\framework\coverage\target\site\xref-test
[DEBUG] -- end configuration --
[WARNING] Unable to locate Test Source XRef to link to - DISABLED
// Cropped for brevity
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.976 s
[INFO] Finished at: 2020-08-04T17:24:57+02:00
[INFO] Final Memory: 13M/32M
[INFO] ------------------------------------------------------------------------

We can clearly see that everything's wrong, from start to end:

• the version is 3.0.0-M5 while I specified 2.22.2;
• the linkXRef parameter is true instead of false;
• the output directory hasn't change either.

Am I missing something on how this plugin's supposed to work?

Answer 1

1. the version is 3.0.0-M5 while I specified 2.22.2

Whole problem is that you run the command mvn surefire-report:report-only and you expect reporting section to be activated. You did nothing but executed the plugin and therefore the section in build/plugins/plugin is activated including the version 2.22.2 and not the reporting section. If you want to activate the reporting section, you have to run the command mvn site and then whole project Page would be generated.

If I was in your position and wanted to generate only Surefire report, then I would use the same configuration in build/plugins/plugin/maven-surefire-report-plugin as you did it in the reporting section now.

    <configuration>
        <linkXRef>false</linkXRef>
        <outputName>test-reports/ut</outputName>
    </configuration>

2. the linkXRef parameter is true instead of false

The same answer as in 1.

3. the output directory hasn't change either

Proper Maven behavior relies on proper reading of documentation, see the link. Please use the parameter outputName instead of outputDirectory in your case (when linkXRef is set to false or override the default Maven property project.reporting.outputDirectory).

Answer 2

I might be mistaken, but I believe there's either a bug here OR the documentation is incorrect. Referencing: http://maven.apache.org/guides/mini/guide-configuring-plugins.html#configuring-reporting-plugins

At time of writing, it reads:

• mvn site

It uses only the parameters defined in the element of each reporting Plugin specified in the element, i.e. site always ignores the parameters defined in the element of each plugin specified in .

• mvn aplugin:areportgoal

It uses firstly the parameters defined in the element of each reporting Plugin specified in the element; if a parameter is not found, it will look up to a parameter defined in the element of each plugin specified in .

Given this configuration

<build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>io.anemortalkid</groupId>
                    <artifactId>sample-report-plugin</artifactId>
                    <version>1.0.0-SNAPSHOT</version>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-site-plugin</artifactId>
                    <version>3.9.0</version>
                </plugin>
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
                <groupId>io.anemortalkid</groupId>
                <artifactId>sample-report-plugin</artifactId>
                <configuration>
                    <word>fromBuild</word>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <reporting>
        <plugins>
            <plugin>
                <groupId>io.anemortalkid</groupId>
                <artifactId>sample-report-plugin</artifactId>
                <configuration>
                    <word>fromReporting</word>
                </configuration>
            </plugin>
        </plugins>
    </reporting>

And the report mojo

@Mojo(name = "word", defaultPhase = LifecyclePhase.SITE, threadSafe = true)
public class SampleReport extends AbstractMavenReport {

  @Parameter private String word;

  protected void executeReport(Locale locale) throws MavenReportException {
    getLog().info("Word is " + word);
  }

  public String getOutputName() {
    return "word.html";
  }

  public String getName(Locale locale) {
    return "word";
  }

  public String getDescription(Locale locale) {
    return "Prints a word from config";
  }
}

The behavior I expect when doing mvn site and mvn sample-report:word to BOTH print:

[INFO] Generating "word" report          --- sample-report-plugin:1.0.0-SNAPSHOT:word
[INFO] Word is fromReporting

Since:

It uses firstly the parameters defined in the element of each reporting Plugin specified in the element; if a parameter is not found, it will look up to a parameter defined in the element of each plugin specified in .

I defined word in reporting, so it should always get the configuration from there.

However the result of invoking the goal directly, contradicts the documentation:

$ mvn sample-report:word
[INFO] Scanning for projects...
[INFO] 
[INFO] --------------------< io.anemortalkid:config-print >--------------------
[INFO] Building config-print 1.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- sample-report-plugin:1.0.0-SNAPSHOT:word (default-cli) @ config-print ---
[INFO] Word is fromBuild
[INFO] ------------------------------------------------------------------------

Reproducible sample: https://github.com/AnEmortalKid/sample-report-plugin

Created https://issues.apache.org/jira/browse/MNG-7006 to confirm

Answer 3

First, use mvn dependency:tree to see if any other plugin has a 3.0.0-M5 dependency to maven-surefire-report-plugin.
If you find one, you might have to setup an exclusion (as in here) in order to force your declaration to hold.
IntelliJ IDEA could also help showing you can which dependencies were suppressed as duplicates.

Check also if there is a profile (possible active by default) which would explain why your configuration is ignored.
I don' t see any obvious bug report on a wrong outputDirectory, so its usage must be off. For instance, as in this example, check your <build> section.