JMH Unable to find the resource: /META-INF/BenchmarkList

Question

I'm not able to run simple JMH benchmark inside eclipse. Maven dependencies:

        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-core</artifactId>
            <version>1.12</version>
        </dependency>
        <dependency>
            <groupId>org.openjdk.jmh</groupId>
            <artifactId>jmh-generator-annprocess</artifactId>
            <version>1.12</version>
        </dependency>

Java code:

public class BTest {
    @Benchmark
    public void test() {
        // todo
    }

    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                 .include(BTest.class.getSimpleName())
                  .build();

        new Runner(opt).run();
    }
}

Result of run:

Exception in thread "main" java.lang.RuntimeException: ERROR: Unable to find the resource: /META-INF/BenchmarkList at org.openjdk.jmh.runner.AbstractResourceReader.getReaders(AbstractResourceReader.java:96) at org.openjdk.jmh.runner.BenchmarkList.find(BenchmarkList.java:104) at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:256) at org.openjdk.jmh.runner.Runner.run(Runner.java:206) at com.test.BTest.main(BTest.java:24)

Maybe the problem is, that I'm running it from eclipse.

Have you tried not running it in eclipse? Since you mentioned it, it would make sense to try it — Will Barnwell, Jun 27 '16 at 14:56
[here](https://github.com/artyushov/idea-jmh-plugin/issues/13#issuecomment-95628473) may be an answer for idea — NanoNova, Jun 21 '20 at 04:57

blacktide · Answer 1 · 2023-01-23T19:25:53.067

If anyone is using Gradle, add the jmh-gradle-plugin to your plugins block:

plugins {
    id 'java'
    id 'me.champeau.jmh' version '0.6.8'
}

Then add all of the following in your dependencies block (check the latest version of JMH on Maven here):

dependencies {
    jmh 'org.openjdk.jmh:jmh-core:1.36'
    jmh 'org.openjdk.jmh:jmh-generator-annprocess:1.36'

    // this is the line that solves the missing /META-INF/BenchmarkList error
    jmhAnnotationProcessor 'org.openjdk.jmh:jmh-generator-annprocess:1.36'
}

Then just use the following to run your benchmarks through Gradle:

./gradlew jmh

Running through your IDE

If you also want to run the benchmarks from within your IDE instead of through Gradle, you can do either of the following:

Option 1 - `main` method

Just use a main method with no additional configuration and your IDE will respect your annotation configuration:

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Fork(value = 1)
@Warmup(iterations = 5, timeUnit = TimeUnit.MILLISECONDS, time = 5000)
@Measurement(iterations = 5, timeUnit = TimeUnit.MILLISECONDS, time = 5000)
public class MyBenchmark {

    public static void main(String[] args) throws RunnerException {
        Options options = new OptionsBuilder()
            .include(MyBenchmark.class.getSimpleName())
            .build();
        new Runner(options).run();
    }

    // benchmarks omitted
}

Option 2 - Install JMH Plugin

If you're using IntelliJ, install the JMH Java Microharness Benchmark Plugin from the Preferences > Plugins section, then you can omit your main method altogether and IntelliJ will give you a run button next to your class name:

`jmhAnnotationProcessor` is not required. If fact, none of the things shown in the `dependencies` block are required unless you want to change the version. — Abhijit Sarkar, Dec 06 '22 at 21:52

score 15 · Accepted Answer · edited Jul 14 '17 at 06:44

Finally found it out. There was a problem with missing exec-maven-plugin plugin

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <executions>
        <execution>
            <id>run-benchmarks</id>
            <phase>integration-test</phase>
            <goals>
                <goal>exec</goal>
            </goals>
            <configuration>
                <classpathScope>test</classpathScope>
                <executable>java</executable>
                <arguments>
                    <argument>-classpath</argument>
                    <classpath />
                    <argument>org.openjdk.jmh.Main</argument>
                    <argument>.*</argument>
                </arguments>
            </configuration>
        </execution>
    </executions>
</plugin>

Weirdly, for Scala + Maven, I had to add this in to even get the error you have above ... : https://github.com/bbarker/scala-maven-jmh-bench-demo/tree/89481477d6d051c038eabbb16ce6c14212844b8d — bbarker, Mar 31 '18 at 02:25

score 8 · Answer 3 · answered Jul 19 '20 at 18:03

pom.xml must have the below dependencies and configurations to Java Micro-benchmark Harness (JMH) Framework

<properties>
    <jmh.version>1.21</jmh.version>
</properties>
<dependencies>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>${jmh.version}</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>${jmh.version}</version>
</dependency>
<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.11</version>
  <scope>test</scope>
</dependency>
</dependencies>
<build>
<finalName>java-jmh</finalName>
<plugins>
    <plugin>    
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
            <source>1.8</source>
            <target>1.8</target>
            <annotationProcessorPaths>
                <path>
                    <groupId>org.openjdk.jmh</groupId>
                    <artifactId>jmh-generator-annprocess</artifactId>
                    <version>${jmh.version}</version>
                </path>
            </annotationProcessorPaths>
        </configuration>
    </plugin>
    <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <executions>
            <execution>
                <id>run-benchmarks</id>
                <phase>integration-test</phase>
                <goals>
                    <goal>exec</goal>
                </goals>
                <configuration>
                    <classpathScope>test</classpathScope>
                    <executable>java</executable>
                    <arguments>
                        <argument>-classpath</argument>
                        <classpath />
                        <argument>org.openjdk.jmh.Main</argument>
                        <argument>.*</argument>
                    </arguments>
                </configuration>
            </execution>
        </executions>
    </plugin>
</plugins>

After this go to command line and run the command $mvn clean install

score 5 · Answer 4 · answered Oct 25 '19 at 09:21

5

This could happen when your compiler plugin has not processed the JMH related annotations. For me, Gill's answer with the maven-compiler-plugin's <annotationProcessorPaths> update worked.

answered Oct 25 '19 at 09:21

rahulkesharwani

328
3
17

score 4 · Answer 5 · answered Nov 28 '18 at 21:39

4

I realized that I already had exec-maven-plugin in my parent pom as mentioned in expected answer but I had to run mvn clean install as mentioned in https://stackoverflow.com/a/40748670 to fix the error

answered Nov 28 '18 at 21:39

ankitkpd

663
5
19

score 4 · Answer 6 · answered Jan 16 '19 at 13:21

4

Having had the same error; and running the tests from maven or intellij didn't work. I realised that the problem was that I wrote the benchmark in Kotlin. Changing the code to java sorted the issue.

answered Jan 16 '19 at 13:21

PeterK

1,697
10
20

2

I banged my head against this error for hours trying to write a `@Benchmark` in Kotlin to no avail. Once I flipped to Java it all worked. – Technetium Aug 01 '22 at 23:30

score 0 · Answer 7 · answered Jul 23 '21 at 04:03

0

Add version as well. This works for me

        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>3.0.0</version>

answered Jul 23 '21 at 04:03

Kravi

289
2
4

score 0 · Answer 8 · answered Mar 23 '23 at 10:14

If you are getting this error using the Intelij Idea's JMH Java Microharness Benchmark Plugin and you also using Gradle try changing the default running task from Gradle to Intelij Idea. To do this:

Settings -> Build, Execution, Deployment -> Build Tools -> Gradle

Change Build and Run using to Intelij IDEA

score 0 · Answer 9 · answered Apr 18 '23 at 11:16

I tried all the above solution. I ended using jmh-maven-plugin:

<plugin>
    <groupId>pw.krejci</groupId>
    <artifactId>jmh-maven-plugin</artifactId>
    <version>0.2.2</version>
</plugin>

and I run it with:

mvn clean install jmh:benchmark

It seems that it does not require exec-maven-plugin, maven-compiler-plugin or maven-shade-plugin.

Notice that the benchmarks must be in the test package.

score 0 · Answer 10 · answered May 25 '23 at 17:21

Here's everything I had to add to pom.xml fix mine.
This snippet ended up pretty close to Ishaq's answer minus a few extras and name-specifics.
I hope this is just a cleaner and easier to readily-use answer.

  <properties>
    <jmh.version>1.36</jmh.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-core</artifactId>
      <version>${jmh.version}</version>
    </dependency>
    <dependency>
      <groupId>org.openjdk.jmh</groupId>
      <artifactId>jmh-generator-annprocess</artifactId>
      <version>${jmh.version}</version>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.2.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <finalName>benchmarks</finalName>
              <transformers>
                <transformer
                        implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                  <mainClass>org.openjdk.jmh.Main</mainClass>
                </transformer>
              </transformers>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

JMH Unable to find the resource: /META-INF/BenchmarkList

10 Answers10

Running through your IDE

Option 1 - `main` method

Option 2 - Install JMH Plugin

Linked

JMH Unable to find the resource: /META-INF/BenchmarkList

10 Answers10

Running through your IDE

Option 1 - main method

Option 2 - Install JMH Plugin

Linked

Option 1 - `main` method