Benchmarking spring boot application with JMH

Question

I have a spring boot application which I want to benchmark using JMH. Any reference for this integration will be useful.

IMHO `jmh` isn't suitable for benchmarking applications but only for particular methods. — Slava Semushin, Jan 03 '17 at 11:27
I manged to do bechmarking for spring MVC application.They should be way to do it for Spring-boot apps — Soumyajit Swain, Jan 03 '17 at 11:43
[spring-cloud-sleuth benchmark samples](https://stackoverflow.com/a/55295621/7509734) — CTD, Mar 22 '19 at 08:35

score 16 · Accepted Answer · edited Jan 17 '20 at 10:15

The solution was quite than easy than I thought. The important part is to start the spring-boot application when the benchmark is getting initialized. Define a class level variable for configuration context and give a reference to it during setup of the benchmark. Make a call to the bean method inside the benchmark.

import java.io.File;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.concurrent.TimeUnit;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.BenchmarkMode;
import org.openjdk.jmh.annotations.Level;
import org.openjdk.jmh.annotations.Mode;
import org.openjdk.jmh.annotations.OutputTimeUnit;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;
import org.springframework.boot.SpringApplication;
import org.springframework.context.ConfigurableApplicationContext;

@BenchmarkMode(Mode.Throughput) @OutputTimeUnit(TimeUnit.MINUTES)
@State(Scope.Thread)
public class ProcessFeedBenchMark   {

    public static void main(String args[]) throws Exception {
        URLClassLoader classLoader = (URLClassLoader) ProcessFeedBenchMark.class.getClassLoader();
        StringBuilder classpath = new StringBuilder();
        for(URL url : classLoader.getURLs())
            classpath.append(url.getPath()).append(File.pathSeparator);
        classpath.append("/D:/work/zymespace/benchmark/src/main/resources/").append(File.pathSeparator);
        System.out.print(classpath.toString());
        System.setProperty("java.class.path", classpath.toString());

        Options opt = new OptionsBuilder()
        .include(ProcessFeedBenchMark.class.getName() + ".*")
        .timeUnit(TimeUnit.MILLISECONDS)
        .threads(1)

        .shouldFailOnError(true)
        .shouldDoGC(true)
        .build();

        new Runner(opt).run();
    }
    static ConfigurableApplicationContext context;

    private BenchmarkTestService service;

    @Setup (Level.Trial) 
    public synchronized void  initialize() {
        try {
            String args = "";
            if(context == null) {
                context = SpringApplication.run(BenchmarkSpringBootStater.class, args );
            }
            service = context.getBean(BenchmarkTestService.class);
            System.out.println(service);
        } catch(Exception e) {
            e.printStackTrace();
        }
    }

    @Benchmark 
    public void benchmark1 (ProcessFeedBenchMark state, Blackhole bh) {
        try {
            service.li();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

I would not rely on a number that you get out of this benchmark. You probably rely on the service configuration and any proxy mechanism. Also, most of the time spend will be for printing the result. Rather, you should return the service value from the method and let JMH handle the escape. This is not how you should use JMH. — Rafael Winterhalter, Jan 09 '17 at 08:26
The System.out.println statement is removed to get the right benchmark for li() method. — Soumyajit Swain, Aug 14 '17 at 18:28
@SoumyajitSwain could you please tell me how you managed to include spring boot fat jar as dependency of JMH project? — VB_, Jun 01 '18 at 15:05
@VolodymyrBakhmatiuk - This line does the trick. System.setProperty("java.class.path", classpath.toString()); — Soumyajit Swain, Jun 04 '18 at 06:57

score 0 · Answer 2 · answered Apr 05 '18 at 02:10

0

There is no need to spin up the Spring context to benchmark a method. In fact, I'm not sure what are you really trying to benchmark here as the method implementation isn't changing. If all you want is to time the method, it's a whole lot simpler and less error-prone to call the method directly. If you want to benchmark the startup time under varying circumstances, see my answer here.

answered Apr 05 '18 at 02:10

Abhijit Sarkar

21,927
20
110
219

2

If you have references to spring beans within your method, it is not possible to directly call the method, otherwise there is no need as you mentioned. – Harsha Jan 24 '19 at 03:35

Benchmarking spring boot application with JMH

2 Answers2